2023-03-14 21:53:35,006 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-14 21:53:35,279 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b 2023-03-14 21:53:35,294 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-14 21:53:35,294 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5, deleteOnExit=true 2023-03-14 21:53:35,294 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-14 21:53:35,295 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/test.cache.data in system properties and HBase conf 2023-03-14 21:53:35,295 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/hadoop.tmp.dir in system properties and HBase conf 2023-03-14 21:53:35,296 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/hadoop.log.dir in system properties and HBase conf 2023-03-14 21:53:35,296 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-14 21:53:35,297 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-14 21:53:35,297 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-14 21:53:35,417 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-14 21:53:35,850 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-14 21:53:35,855 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-14 21:53:35,855 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-14 21:53:35,855 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-14 21:53:35,856 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-14 21:53:35,856 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-14 21:53:35,856 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-14 21:53:35,856 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-14 21:53:35,857 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-14 21:53:35,857 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-14 21:53:35,858 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/nfs.dump.dir in system properties and HBase conf 2023-03-14 21:53:35,858 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/java.io.tmpdir in system properties and HBase conf 2023-03-14 21:53:35,859 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-14 21:53:35,859 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-14 21:53:35,859 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-14 21:53:36,422 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-14 21:53:36,426 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-14 21:53:36,749 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-14 21:53:36,945 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-14 21:53:36,960 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-14 21:53:37,001 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-14 21:53:37,035 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/java.io.tmpdir/Jetty_localhost_45079_hdfs____.ngfwit/webapp 2023-03-14 21:53:37,185 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45079 2023-03-14 21:53:37,225 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-14 21:53:37,225 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-14 21:53:37,686 WARN [Listener at localhost/39547] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-14 21:53:37,823 WARN [Listener at localhost/39547] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-14 21:53:37,847 WARN [Listener at localhost/39547] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-14 21:53:37,855 INFO [Listener at localhost/39547] log.Slf4jLog(67): jetty-6.1.26 2023-03-14 21:53:37,862 INFO [Listener at localhost/39547] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/java.io.tmpdir/Jetty_localhost_44731_datanode____.i3t16s/webapp 2023-03-14 21:53:37,980 INFO [Listener at localhost/39547] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44731 2023-03-14 21:53:38,397 WARN [Listener at localhost/45975] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-14 21:53:39,029 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x218ad51a850afb7b: Processing first storage report for DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870 from datanode ea84138e-3091-4fa6-8b1e-81f8e4e08e46 2023-03-14 21:53:39,031 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x218ad51a850afb7b: from storage DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870 node DatanodeRegistration(127.0.0.1:39401, datanodeUuid=ea84138e-3091-4fa6-8b1e-81f8e4e08e46, infoPort=44113, infoSecurePort=0, ipcPort=45975, storageInfo=lv=-57;cid=testClusterID;nsid=1063125760;c=1678830816501), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-14 21:53:39,032 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x218ad51a850afb7b: Processing first storage report for DS-28ce46ed-deb3-486b-8206-e49576ea5f58 from datanode ea84138e-3091-4fa6-8b1e-81f8e4e08e46 2023-03-14 21:53:39,032 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x218ad51a850afb7b: from storage DS-28ce46ed-deb3-486b-8206-e49576ea5f58 node DatanodeRegistration(127.0.0.1:39401, datanodeUuid=ea84138e-3091-4fa6-8b1e-81f8e4e08e46, infoPort=44113, infoSecurePort=0, ipcPort=45975, storageInfo=lv=-57;cid=testClusterID;nsid=1063125760;c=1678830816501), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-14 21:53:39,119 DEBUG [Listener at localhost/45975] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b 2023-03-14 21:53:39,177 INFO [Listener at localhost/45975] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/zookeeper_0, clientPort=58882, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-14 21:53:39,190 INFO [Listener at localhost/45975] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58882 2023-03-14 21:53:39,199 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:39,202 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:39,881 INFO [Listener at localhost/45975] util.FSUtils(471): Created version file at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63 with version=8 2023-03-14 21:53:39,882 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/hbase-staging 2023-03-14 21:53:40,176 INFO [Listener at localhost/45975] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-14 21:53:40,636 INFO [Listener at localhost/45975] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-14 21:53:40,673 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:40,674 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:40,674 INFO [Listener at localhost/45975] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-14 21:53:40,674 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:40,675 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-14 21:53:40,829 INFO [Listener at localhost/45975] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-14 21:53:40,903 DEBUG [Listener at localhost/45975] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-14 21:53:40,996 INFO [Listener at localhost/45975] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40679 2023-03-14 21:53:41,006 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:41,008 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:41,030 INFO [Listener at localhost/45975] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40679 connecting to ZooKeeper ensemble=127.0.0.1:58882 2023-03-14 21:53:41,070 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:406790x0, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-14 21:53:41,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:40679-0x100e50541210000 connected 2023-03-14 21:53:41,101 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-14 21:53:41,101 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-14 21:53:41,105 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-14 21:53:41,113 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40679 2023-03-14 21:53:41,113 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40679 2023-03-14 21:53:41,114 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40679 2023-03-14 21:53:41,116 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40679 2023-03-14 21:53:41,116 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40679 2023-03-14 21:53:41,123 INFO [Listener at localhost/45975] master.HMaster(444): hbase.rootdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63, hbase.cluster.distributed=false 2023-03-14 21:53:41,198 INFO [Listener at localhost/45975] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-14 21:53:41,198 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:41,199 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:41,199 INFO [Listener at localhost/45975] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-14 21:53:41,199 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-14 21:53:41,199 INFO [Listener at localhost/45975] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-14 21:53:41,204 INFO [Listener at localhost/45975] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-14 21:53:41,207 INFO [Listener at localhost/45975] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35143 2023-03-14 21:53:41,209 INFO [Listener at localhost/45975] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-14 21:53:41,215 DEBUG [Listener at localhost/45975] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-14 21:53:41,216 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:41,219 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:41,220 INFO [Listener at localhost/45975] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35143 connecting to ZooKeeper ensemble=127.0.0.1:58882 2023-03-14 21:53:41,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:351430x0, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-14 21:53:41,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35143-0x100e50541210001 connected 2023-03-14 21:53:41,225 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-14 21:53:41,226 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-14 21:53:41,227 DEBUG [Listener at localhost/45975] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-14 21:53:41,227 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35143 2023-03-14 21:53:41,228 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35143 2023-03-14 21:53:41,229 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35143 2023-03-14 21:53:41,229 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35143 2023-03-14 21:53:41,230 DEBUG [Listener at localhost/45975] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35143 2023-03-14 21:53:41,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:41,242 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-14 21:53:41,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:41,263 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-14 21:53:41,263 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-14 21:53:41,264 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:41,265 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-14 21:53:41,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,40679,1678830820017 from backup master directory 2023-03-14 21:53:41,266 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-14 21:53:41,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:41,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-14 21:53:41,274 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-14 21:53:41,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:41,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-14 21:53:41,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-14 21:53:41,774 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/hbase.id with ID: 82b669a3-8f61-415e-b168-033a50fab02f 2023-03-14 21:53:41,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:41,831 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:41,868 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x664ce928 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:41,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3974be55, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:41,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-14 21:53:41,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-14 21:53:41,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-14 21:53:41,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-14 21:53:41,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-14 21:53:41,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-14 21:53:41,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-14 21:53:41,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store-tmp 2023-03-14 21:53:42,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:42,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-14 21:53:42,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:53:42,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:53:42,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 1 ms 2023-03-14 21:53:42,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:53:42,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:53:42,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-14 21:53:42,023 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/WALs/jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:42,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40679%2C1678830820017, suffix=, logDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/WALs/jenkins-hbase4.apache.org,40679,1678830820017, archiveDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/oldWALs, maxLogs=10 2023-03-14 21:53:42,114 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK] 2023-03-14 21:53:42,123 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-14 21:53:42,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/WALs/jenkins-hbase4.apache.org,40679,1678830820017/jenkins-hbase4.apache.org%2C40679%2C1678830820017.1678830822066 2023-03-14 21:53:42,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK]] 2023-03-14 21:53:42,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:42,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:42,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,263 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,271 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-14 21:53:42,300 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-14 21:53:42,315 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:42,321 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-14 21:53:42,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:53:42,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74940666, jitterRate=0.11670294404029846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:53:42,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-14 21:53:42,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-14 21:53:42,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-14 21:53:42,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-14 21:53:42,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-14 21:53:42,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-14 21:53:42,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-03-14 21:53:42,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-14 21:53:42,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-14 21:53:42,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-14 21:53:42,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-14 21:53:42,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-14 21:53:42,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-14 21:53:42,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-14 21:53:42,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-14 21:53:42,490 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:42,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-14 21:53:42,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-14 21:53:42,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-14 21:53:42,511 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-14 21:53:42,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-14 21:53:42,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:42,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40679,1678830820017, sessionid=0x100e50541210000, setting cluster-up flag (Was=false) 2023-03-14 21:53:42,530 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:42,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-14 21:53:42,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:42,562 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:42,569 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-14 21:53:42,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:42,574 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp 2023-03-14 21:53:42,637 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(952): ClusterId : 82b669a3-8f61-415e-b168-033a50fab02f 2023-03-14 21:53:42,641 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-14 21:53:42,650 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-14 21:53:42,650 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-14 21:53:42,655 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-14 21:53:42,656 DEBUG [RS:0;jenkins-hbase4:35143] zookeeper.ReadOnlyZKClient(139): Connect 0x28c65a01 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:42,668 DEBUG [RS:0;jenkins-hbase4:35143] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2601b169, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:42,669 DEBUG [RS:0;jenkins-hbase4:35143] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@223e3171, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-14 21:53:42,700 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35143 2023-03-14 21:53:42,705 INFO [RS:0;jenkins-hbase4:35143] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-14 21:53:42,705 INFO [RS:0;jenkins-hbase4:35143] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-14 21:53:42,706 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1023): About to register with Master. 2023-03-14 21:53:42,709 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,40679,1678830820017 with isa=jenkins-hbase4.apache.org/172.31.14.131:35143, startcode=1678830821197 2023-03-14 21:53:42,719 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-14 21:53:42,731 DEBUG [RS:0;jenkins-hbase4:35143] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-14 21:53:42,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-14 21:53:42,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-14 21:53:42,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-14 21:53:42,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-14 21:53:42,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-14 21:53:42,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-14 21:53:42,733 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678830852736 2023-03-14 21:53:42,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-14 21:53:42,741 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-14 21:53:42,741 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-14 21:53:42,747 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-14 21:53:42,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-14 21:53:42,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-14 21:53:42,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-14 21:53:42,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-14 21:53:42,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-14 21:53:42,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-14 21:53:42,773 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-14 21:53:42,773 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-14 21:53:42,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-14 21:53:42,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-14 21:53:42,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678830822778,5,FailOnTimeoutGroup] 2023-03-14 21:53:42,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678830822779,5,FailOnTimeoutGroup] 2023-03-14 21:53:42,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-14 21:53:42,782 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,800 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45757, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-14 21:53:42,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,836 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63 2023-03-14 21:53:42,837 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:39547 2023-03-14 21:53:42,837 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-14 21:53:42,844 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-14 21:53:42,846 DEBUG [RS:0;jenkins-hbase4:35143] zookeeper.ZKUtil(162): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,846 WARN [RS:0;jenkins-hbase4:35143] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-14 21:53:42,846 INFO [RS:0;jenkins-hbase4:35143] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-14 21:53:42,847 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1947): logDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/WALs/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,847 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35143,1678830821197] 2023-03-14 21:53:42,857 DEBUG [RS:0;jenkins-hbase4:35143] zookeeper.ZKUtil(162): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,869 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-14 21:53:42,878 INFO [RS:0;jenkins-hbase4:35143] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-14 21:53:42,902 INFO [RS:0;jenkins-hbase4:35143] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-14 21:53:42,907 INFO [RS:0;jenkins-hbase4:35143] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:53:42,907 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,909 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-14 21:53:42,916 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,916 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-14 21:53:42,917 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,918 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,918 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,918 DEBUG [RS:0;jenkins-hbase4:35143] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-14 21:53:42,925 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,926 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,926 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,946 INFO [RS:0;jenkins-hbase4:35143] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-14 21:53:42,950 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35143,1678830821197-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:42,967 INFO [RS:0;jenkins-hbase4:35143] regionserver.Replication(203): jenkins-hbase4.apache.org,35143,1678830821197 started 2023-03-14 21:53:42,967 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,35143,1678830821197, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35143, sessionid=0x100e50541210001 2023-03-14 21:53:42,967 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-14 21:53:42,967 DEBUG [RS:0;jenkins-hbase4:35143] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,967 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35143,1678830821197' 2023-03-14 21:53:42,967 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:42,968 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:42,968 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-14 21:53:42,969 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-14 21:53:42,969 DEBUG [RS:0;jenkins-hbase4:35143] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:42,969 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35143,1678830821197' 2023-03-14 21:53:42,969 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-14 21:53:42,969 DEBUG [RS:0;jenkins-hbase4:35143] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-14 21:53:42,970 DEBUG [RS:0;jenkins-hbase4:35143] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-14 21:53:42,970 INFO [RS:0;jenkins-hbase4:35143] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-14 21:53:42,970 INFO [RS:0;jenkins-hbase4:35143] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-14 21:53:43,083 INFO [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35143%2C1678830821197, suffix=, logDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/WALs/jenkins-hbase4.apache.org,35143,1678830821197, archiveDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/oldWALs, maxLogs=32 2023-03-14 21:53:43,106 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK] 2023-03-14 21:53:43,109 INFO [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/WALs/jenkins-hbase4.apache.org,35143,1678830821197/jenkins-hbase4.apache.org%2C35143%2C1678830821197.1678830823086 2023-03-14 21:53:43,110 DEBUG [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK]] 2023-03-14 21:53:43,208 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:43,209 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:43,209 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63 2023-03-14 21:53:43,624 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:43,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-14 21:53:43,630 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/info 2023-03-14 21:53:43,631 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-14 21:53:43,632 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:43,632 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-14 21:53:43,635 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/rep_barrier 2023-03-14 21:53:43,636 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-14 21:53:43,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:43,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-14 21:53:43,639 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/table 2023-03-14 21:53:43,640 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-14 21:53:43,641 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:43,643 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740 2023-03-14 21:53:43,644 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740 2023-03-14 21:53:43,648 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-14 21:53:43,650 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-14 21:53:43,654 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:53:43,655 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69126509, jitterRate=0.030065253376960754}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-14 21:53:43,655 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-14 21:53:43,655 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-14 21:53:43,655 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-14 21:53:43,655 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-14 21:53:43,655 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-14 21:53:43,655 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-14 21:53:43,656 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-14 21:53:43,656 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-14 21:53:43,661 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-14 21:53:43,661 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-14 21:53:43,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-14 21:53:43,685 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-14 21:53:43,688 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-14 21:53:43,840 DEBUG [jenkins-hbase4:40679] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-14 21:53:43,843 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35143,1678830821197, state=OPENING 2023-03-14 21:53:43,852 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-14 21:53:43,853 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:43,854 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-14 21:53:43,856 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:44,038 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:44,041 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-14 21:53:44,044 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35230, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-14 21:53:44,056 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-14 21:53:44,057 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-14 21:53:44,060 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35143%2C1678830821197.meta, suffix=.meta, logDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/WALs/jenkins-hbase4.apache.org,35143,1678830821197, archiveDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/oldWALs, maxLogs=32 2023-03-14 21:53:44,077 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK] 2023-03-14 21:53:44,084 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/WALs/jenkins-hbase4.apache.org,35143,1678830821197/jenkins-hbase4.apache.org%2C35143%2C1678830821197.meta.1678830824061.meta 2023-03-14 21:53:44,084 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39401,DS-6adcde67-f65b-4a20-b10c-8fe6bedc7870,DISK]] 2023-03-14 21:53:44,084 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:44,086 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-14 21:53:44,105 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-14 21:53:44,110 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-14 21:53:44,116 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-14 21:53:44,116 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:44,116 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-14 21:53:44,116 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-14 21:53:44,120 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-14 21:53:44,122 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/info 2023-03-14 21:53:44,122 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/info 2023-03-14 21:53:44,123 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-14 21:53:44,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:44,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-14 21:53:44,125 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/rep_barrier 2023-03-14 21:53:44,126 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/rep_barrier 2023-03-14 21:53:44,126 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-14 21:53:44,127 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:44,127 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-14 21:53:44,129 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/table 2023-03-14 21:53:44,129 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/table 2023-03-14 21:53:44,129 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-14 21:53:44,130 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:44,132 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740 2023-03-14 21:53:44,135 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740 2023-03-14 21:53:44,139 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-14 21:53:44,142 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-14 21:53:44,143 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60427413, jitterRate=-0.09956137835979462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-14 21:53:44,143 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-14 21:53:44,153 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678830824031 2023-03-14 21:53:44,170 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-14 21:53:44,171 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-14 21:53:44,171 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35143,1678830821197, state=OPEN 2023-03-14 21:53:44,174 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-14 21:53:44,174 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-14 21:53:44,178 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-14 21:53:44,178 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35143,1678830821197 in 318 msec 2023-03-14 21:53:44,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-14 21:53:44,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 509 msec 2023-03-14 21:53:44,187 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5550 sec 2023-03-14 21:53:44,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678830824187, completionTime=-1 2023-03-14 21:53:44,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-14 21:53:44,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-14 21:53:44,272 DEBUG [hconnection-0x734e5f2d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:53:44,277 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35238, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:53:44,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-14 21:53:44,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678830884297 2023-03-14 21:53:44,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678830944297 2023-03-14 21:53:44,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 109 msec 2023-03-14 21:53:44,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40679,1678830820017-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:44,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40679,1678830820017-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:44,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40679,1678830820017-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:44,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40679, period=300000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:44,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-14 21:53:44,330 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-14 21:53:44,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-14 21:53:44,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-14 21:53:44,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-14 21:53:44,350 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:53:44,353 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:53:44,375 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:44,377 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b empty. 2023-03-14 21:53:44,378 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:44,378 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-14 21:53:44,404 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:44,407 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => cdbd6374c3e2fe4a973c440ad92e967b, NAME => 'hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:53:44,820 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:44,820 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing cdbd6374c3e2fe4a973c440ad92e967b, disabling compactions & flushes 2023-03-14 21:53:44,820 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:44,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:44,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. after waiting 0 ms 2023-03-14 21:53:44,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:44,821 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:44,821 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for cdbd6374c3e2fe4a973c440ad92e967b: 2023-03-14 21:53:44,826 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:53:44,842 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678830824828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830824828"}]},"ts":"1678830824828"} 2023-03-14 21:53:44,872 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:53:44,875 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:53:44,880 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830824875"}]},"ts":"1678830824875"} 2023-03-14 21:53:44,885 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-14 21:53:44,896 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cdbd6374c3e2fe4a973c440ad92e967b, ASSIGN}] 2023-03-14 21:53:44,899 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=cdbd6374c3e2fe4a973c440ad92e967b, ASSIGN 2023-03-14 21:53:44,901 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=cdbd6374c3e2fe4a973c440ad92e967b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:53:45,052 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=cdbd6374c3e2fe4a973c440ad92e967b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:45,052 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678830825051"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830825051"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830825051"}]},"ts":"1678830825051"} 2023-03-14 21:53:45,056 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure cdbd6374c3e2fe4a973c440ad92e967b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:45,217 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:45,217 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cdbd6374c3e2fe4a973c440ad92e967b, NAME => 'hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:45,218 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,218 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:45,218 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,219 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,220 INFO [StoreOpener-cdbd6374c3e2fe4a973c440ad92e967b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,223 DEBUG [StoreOpener-cdbd6374c3e2fe4a973c440ad92e967b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/info 2023-03-14 21:53:45,223 DEBUG [StoreOpener-cdbd6374c3e2fe4a973c440ad92e967b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/info 2023-03-14 21:53:45,223 INFO [StoreOpener-cdbd6374c3e2fe4a973c440ad92e967b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cdbd6374c3e2fe4a973c440ad92e967b columnFamilyName info 2023-03-14 21:53:45,224 INFO [StoreOpener-cdbd6374c3e2fe4a973c440ad92e967b-1] regionserver.HStore(310): Store=cdbd6374c3e2fe4a973c440ad92e967b/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:45,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,226 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,231 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:53:45,234 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:53:45,235 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cdbd6374c3e2fe4a973c440ad92e967b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60216701, jitterRate=-0.10270123183727264}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:53:45,235 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cdbd6374c3e2fe4a973c440ad92e967b: 2023-03-14 21:53:45,237 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b., pid=6, masterSystemTime=1678830825210 2023-03-14 21:53:45,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:45,241 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:53:45,242 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=cdbd6374c3e2fe4a973c440ad92e967b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:45,242 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678830825241"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830825241"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830825241"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830825241"}]},"ts":"1678830825241"} 2023-03-14 21:53:45,250 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-14 21:53:45,251 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure cdbd6374c3e2fe4a973c440ad92e967b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 191 msec 2023-03-14 21:53:45,254 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-14 21:53:45,254 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=cdbd6374c3e2fe4a973c440ad92e967b, ASSIGN in 354 msec 2023-03-14 21:53:45,255 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:53:45,256 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830825256"}]},"ts":"1678830825256"} 2023-03-14 21:53:45,258 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-14 21:53:45,263 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:53:45,265 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 923 msec 2023-03-14 21:53:45,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-14 21:53:45,353 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-14 21:53:45,353 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:53:45,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-14 21:53:45,406 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-14 21:53:45,412 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-03-14 21:53:45,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-14 21:53:45,431 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-14 21:53:45,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-03-14 21:53:45,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-14 21:53:45,449 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-14 21:53:45,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.174sec 2023-03-14 21:53:45,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-14 21:53:45,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-14 21:53:45,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-14 21:53:45,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40679,1678830820017-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-14 21:53:45,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40679,1678830820017-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-14 21:53:45,466 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-14 21:53:45,543 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(139): Connect 0x0629018b to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:45,548 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c1e7f16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:45,562 DEBUG [hconnection-0x77bc4ff1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:53:45,572 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35242, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:53:45,583 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:53:45,583 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(139): Connect 0x7defec53 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:45,589 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64b944ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:45,589 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:53:45,611 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=169, AvailableMemoryMB=3572 2023-03-14 21:53:45,622 DEBUG [Listener at localhost/45975] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:53:45,625 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57106, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:53:45,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:53:45,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:45,641 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:53:45,643 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:53:45,645 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:45,646 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d empty. 2023-03-14 21:53:45,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-14 21:53:45,648 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:45,648 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-14 21:53:45,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-14 21:53:45,692 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:45,694 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 664facdeb00d26e025c40b36746b659d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:53:45,713 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:45,713 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 664facdeb00d26e025c40b36746b659d, disabling compactions & flushes 2023-03-14 21:53:45,713 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:45,713 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:45,713 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. after waiting 0 ms 2023-03-14 21:53:45,714 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:45,714 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:45,714 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:45,720 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:53:45,721 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830825721"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830825721"}]},"ts":"1678830825721"} 2023-03-14 21:53:45,725 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:53:45,726 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:53:45,726 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830825726"}]},"ts":"1678830825726"} 2023-03-14 21:53:45,729 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-14 21:53:45,735 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, ASSIGN}] 2023-03-14 21:53:45,737 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, ASSIGN 2023-03-14 21:53:45,739 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:53:45,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-14 21:53:45,891 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:45,891 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830825891"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830825891"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830825891"}]},"ts":"1678830825891"} 2023-03-14 21:53:45,894 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:45,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-14 21:53:46,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:46,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 664facdeb00d26e025c40b36746b659d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:46,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:46,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,057 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,059 DEBUG [StoreOpener-664facdeb00d26e025c40b36746b659d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:46,059 DEBUG [StoreOpener-664facdeb00d26e025c40b36746b659d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:46,062 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 664facdeb00d26e025c40b36746b659d columnFamilyName family1 2023-03-14 21:53:46,065 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] regionserver.HStore(310): Store=664facdeb00d26e025c40b36746b659d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:46,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:46,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:53:46,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 664facdeb00d26e025c40b36746b659d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73969902, jitterRate=0.10223743319511414}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:53:46,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:46,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., pid=11, masterSystemTime=1678830826047 2023-03-14 21:53:46,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:46,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:46,085 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:46,085 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830826085"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830826085"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830826085"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830826085"}]},"ts":"1678830826085"} 2023-03-14 21:53:46,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-14 21:53:46,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197 in 195 msec 2023-03-14 21:53:46,096 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-14 21:53:46,097 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, ASSIGN in 359 msec 2023-03-14 21:53:46,098 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:53:46,098 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830826098"}]},"ts":"1678830826098"} 2023-03-14 21:53:46,101 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-14 21:53:46,104 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:53:46,107 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 467 msec 2023-03-14 21:53:46,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-14 21:53:46,297 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-14 21:53:46,314 DEBUG [hconnection-0x130f47b7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:53:46,318 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35246, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:53:46,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:46,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,495 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:46,495 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:46,496 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:46,496 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:46,497 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,497 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,498 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:46,499 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:46,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:46,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:46,499 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:46,499 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:46,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:46,503 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:46,503 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:46,505 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:46,506 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:46,507 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:53:47,051 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307b7e55e16ec748bcb9e27a498f8d0d63 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307b7e55e16ec748bcb9e27a498f8d0d63 2023-03-14 21:53:47,466 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/39407c48d187460c96c11675fedaea8f 2023-03-14 21:53:47,476 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/39407c48d187460c96c11675fedaea8f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f 2023-03-14 21:53:47,484 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-14 21:53:47,487 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 980ms, sequenceid=5, compaction requested=false 2023-03-14 21:53:47,488 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:47,491 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:47,491 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:47,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:47,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:47,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:47,494 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,494 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:47,494 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,494 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:47,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:47,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:47,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:47,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@262eccb7[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:47,496 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:47,497 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,499 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:47,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:47,499 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:47,499 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:47,502 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:47,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:47,505 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:47,505 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:47,505 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:47,506 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,506 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:47,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:47,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:47,509 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,510 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,517 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,517 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:47,517 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:47,517 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:47,517 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:47,517 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:47,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:47,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:47,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:47,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:47,520 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:47,520 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:47,620 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:47,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:47,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:47,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:47,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,639 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:47,639 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:47,640 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:47,640 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:47,640 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,640 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,643 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:47,643 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:47,644 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:47,644 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:47,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:47,647 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:47,647 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:47,647 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:53:48,071 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d47b06326e11424481339f1296deecb6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128d47b06326e11424481339f1296deecb6 2023-03-14 21:53:48,481 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/cb0f1cb83ce44c4097add2c84a33c777 2023-03-14 21:53:48,490 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/cb0f1cb83ce44c4097add2c84a33c777 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777 2023-03-14 21:53:48,497 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:53:48,498 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 851ms, sequenceid=9, compaction requested=false 2023-03-14 21:53:48,499 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:48,499 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:48,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:48,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:48,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:48,502 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:48,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39e24248[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:48,504 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:48,504 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,505 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:48,506 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,506 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:48,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:48,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:48,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:48,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:48,511 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:48,511 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:48,511 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:48,511 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,511 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:48,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,518 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:48,518 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,520 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,524 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,524 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:48,524 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:48,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:48,524 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:48,524 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:48,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:48,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:48,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:48,524 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:48,525 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:48,625 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:48,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:48,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:48,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:48,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,637 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:48,638 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:48,638 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:48,638 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:48,639 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,639 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,640 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:48,640 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:48,641 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,641 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:48,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:48,643 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:48,643 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:48,643 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:48,643 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:48,643 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:48,643 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:48,675 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f4e87121b9754de0bca5e9afd0e07613 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f4e87121b9754de0bca5e9afd0e07613 2023-03-14 21:53:48,693 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/b8f23bd84b004806a670602ff9bda5fc 2023-03-14 21:53:48,703 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/b8f23bd84b004806a670602ff9bda5fc as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc 2023-03-14 21:53:48,712 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc, entries=1, sequenceid=13, filesize=5.0 K 2023-03-14 21:53:48,713 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 70ms, sequenceid=13, compaction requested=true 2023-03-14 21:53:48,714 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:48,714 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:48,714 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:48,715 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:48,715 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,715 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:48,715 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:48,717 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,717 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,718 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,718 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:48,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:48,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ee5d9f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:48,720 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:48,720 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,722 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,722 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:48,722 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:48,722 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:48,723 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,723 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:48,724 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:48,725 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,725 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:48,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:48,729 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:48,730 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:48,730 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:48,730 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,730 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:48,733 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,733 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,733 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:48,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:48,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:48,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:48,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:48,734 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:48,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,736 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:48,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,741 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:48,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:48,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:48,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:48,742 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:48,743 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:48,745 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:48,745 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:48,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:48,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,750 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,843 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:48,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:48,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:48,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:48,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,860 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:48,860 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:48,860 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:48,860 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:48,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,863 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:48,863 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:48,863 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:48,863 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:48,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,865 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,865 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:48,865 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:48,865 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:48,865 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:48,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:48,866 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:48,866 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:48,866 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:48,874 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-14 21:53:48,876 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-14 21:53:48,901 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055eb5ac9008304f3a9e937bbb60cf93d1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055eb5ac9008304f3a9e937bbb60cf93d1 2023-03-14 21:53:49,314 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/4d1a9ad85008471ebc17f36c24e4de58 2023-03-14 21:53:49,326 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/4d1a9ad85008471ebc17f36c24e4de58 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58 2023-03-14 21:53:49,337 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58, entries=1, sequenceid=17, filesize=5.0 K 2023-03-14 21:53:49,339 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 473ms, sequenceid=17, compaction requested=true 2023-03-14 21:53:49,339 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:49,339 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:49,339 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:49,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:49,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:49,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:49,343 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:49,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51995903[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:49,347 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:49,347 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:49,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:49,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:49,349 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,349 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:49,351 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,351 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:49,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,351 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:49,352 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:49,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:49,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:49,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:49,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:49,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:49,360 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,360 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,360 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:49,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:49,361 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,363 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,368 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,368 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:49,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:49,368 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:49,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:49,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:49,368 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,369 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,369 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,369 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,369 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,369 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,368 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:49,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:49,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:49,370 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:49,370 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:49,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:49,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:49,470 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:49,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:49,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:49,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:49,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,484 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:49,484 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:49,485 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:49,485 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:49,486 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,486 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,488 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:49,488 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:49,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:49,489 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,489 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:49,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,490 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:49,490 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:49,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:49,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:49,491 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:49,491 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:49,491 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:49,491 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:49,515 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bf8da4f9c3294930b212c91cd843b88e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf8da4f9c3294930b212c91cd843b88e 2023-03-14 21:53:49,647 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:53:49,926 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/ff0e8d69bd1946a7900263a89c6ab277 2023-03-14 21:53:49,935 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/ff0e8d69bd1946a7900263a89c6ab277 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277 2023-03-14 21:53:49,943 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277, entries=1, sequenceid=21, filesize=5.0 K 2023-03-14 21:53:49,944 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 453ms, sequenceid=21, compaction requested=true 2023-03-14 21:53:49,944 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:49,944 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:49,944 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:49,949 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,950 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,950 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:49,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:49,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1027cf3[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:49,952 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:49,952 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,953 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:49,953 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,954 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:49,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:49,955 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:49,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:49,958 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:49,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:49,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:49,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,958 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:49,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:49,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:49,962 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:49,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,963 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:49,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:49,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:49,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:49,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:49,966 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:49,966 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:49,967 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:49,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:49,967 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:49,967 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:50,068 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:50,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:50,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:50,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:50,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,079 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:50,079 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:50,080 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:50,080 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:50,080 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,080 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,082 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,082 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:50,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:50,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:50,082 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,082 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:50,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:50,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:50,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:50,084 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:50,084 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:50,085 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:50,170 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-14 21:53:50,171 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-14 21:53:50,507 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c32bb9f5e99048d4a6ee76ca643a3dd9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231c32bb9f5e99048d4a6ee76ca643a3dd9 2023-03-14 21:53:50,918 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/506f5c566d2e47d888cfbc652fe977d8 2023-03-14 21:53:50,928 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/506f5c566d2e47d888cfbc652fe977d8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8 2023-03-14 21:53:50,937 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8, entries=1, sequenceid=25, filesize=5.0 K 2023-03-14 21:53:50,939 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 854ms, sequenceid=25, compaction requested=true 2023-03-14 21:53:50,939 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:50,939 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:50,939 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:50,942 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:50,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:50,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a598e99[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:50,945 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:50,945 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,946 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,947 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:50,947 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:50,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:50,947 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:50,949 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:50,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:50,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:50,953 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:50,953 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:50,953 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:50,953 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,953 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:50,955 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,955 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,955 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:50,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:50,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:50,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,956 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:50,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:50,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,957 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:50,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:50,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:50,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:50,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:50,961 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:50,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:50,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:50,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:50,962 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:50,962 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:50,962 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:50,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:50,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:51,062 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:51,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:51,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:51,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:51,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,073 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:51,073 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:51,074 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:51,074 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:51,074 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,074 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,076 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:51,076 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:51,076 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,076 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:51,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:51,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:51,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:51,079 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:51,079 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:51,079 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:51,079 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:51,079 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:51,505 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010179a08fe17ecd41d99f14fe66c7305122 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010179a08fe17ecd41d99f14fe66c7305122 2023-03-14 21:53:51,916 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/9d65a823c22b4aa881d627ce14183115 2023-03-14 21:53:51,925 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/9d65a823c22b4aa881d627ce14183115 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115 2023-03-14 21:53:51,933 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115, entries=1, sequenceid=29, filesize=5.0 K 2023-03-14 21:53:51,935 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 856ms, sequenceid=29, compaction requested=true 2023-03-14 21:53:51,935 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:51,936 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:51,936 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:51,936 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:51,936 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,936 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:51,936 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:51,938 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:51,939 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,939 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:51,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:51,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:51,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:51,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e2f3ce8[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:51,942 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:51,942 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,943 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,943 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:51,944 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,944 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:51,946 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:51,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:51,950 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:51,950 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:51,950 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:51,950 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,950 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:51,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:51,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:51,953 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,955 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:51,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:51,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:51,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:51,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:51,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:51,961 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:51,961 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:51,962 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:51,962 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:51,962 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,963 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,963 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:51,962 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:52,063 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:52,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:52,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:52,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:52,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,088 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:52,088 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:52,088 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:52,088 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:52,089 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,089 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,091 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:52,091 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:52,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:52,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,092 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,092 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:52,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:52,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,093 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:52,093 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:52,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:52,096 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:52,096 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:52,096 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:52,096 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:52,096 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:52,567 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039815af23549f4d8493fe294aa1bfc01f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039815af23549f4d8493fe294aa1bfc01f 2023-03-14 21:53:52,582 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/6141b73e966f4d3b85f9915c653010d1 2023-03-14 21:53:52,591 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/6141b73e966f4d3b85f9915c653010d1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1 2023-03-14 21:53:52,599 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1, entries=1, sequenceid=33, filesize=5.0 K 2023-03-14 21:53:52,601 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 505ms, sequenceid=33, compaction requested=true 2023-03-14 21:53:52,601 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:52,601 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:52,601 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:52,602 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:52,602 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,602 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:52,602 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:52,603 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,603 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:52,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:52,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4565376f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:52,606 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:52,606 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,608 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:52,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:52,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:52,609 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,609 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:52,610 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:52,610 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:52,611 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:52,614 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:52,615 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:52,615 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:52,615 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,615 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:52,623 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,623 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,623 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:52,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:52,624 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:52,628 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,630 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,630 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,630 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,630 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:52,631 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:52,631 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:52,631 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:52,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:52,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:52,631 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,632 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,632 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:52,632 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:52,632 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:52,632 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:52,732 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:52,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:52,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:52,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:52,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,744 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:52,744 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:52,745 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:52,745 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:52,746 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,746 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,748 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:52,748 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:52,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:52,748 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:52,748 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:52,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:52,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:52,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:52,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:52,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:52,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:52,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:52,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:52,752 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:52,752 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:52,753 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:53,173 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078fb722939c6f4d99a254b606d3ad1050 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509078fb722939c6f4d99a254b606d3ad1050 2023-03-14 21:53:53,582 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/d37ff076c52045fa8be3e963cb796453 2023-03-14 21:53:53,590 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/d37ff076c52045fa8be3e963cb796453 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453 2023-03-14 21:53:53,598 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453, entries=1, sequenceid=37, filesize=5.0 K 2023-03-14 21:53:53,600 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 847ms, sequenceid=37, compaction requested=true 2023-03-14 21:53:53,600 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:53,600 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:53,600 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:53,602 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,602 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:53,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:53,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57922ed4[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:53,605 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:53,605 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,607 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:53,607 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,607 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:53,609 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:53,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:53,609 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:53,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:53,612 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:53,612 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:53,612 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:53,612 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,612 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:53,614 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,614 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:53,614 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,615 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:53,616 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,620 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,620 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:53,620 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:53,620 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:53,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:53,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:53,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:53,620 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:53,620 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:53,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:53,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:53,621 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:53,721 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:53,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:53,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:53:53,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-14 21:53:53,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,732 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:53,732 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:53,732 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-14 21:53:53,732 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-14 21:53:53,733 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,733 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,736 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:53,736 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:53,736 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:53,736 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:53:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:53:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-14 21:53:53,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:53,738 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-14 21:53:53,738 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:53:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-14 21:53:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:53:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:53:53,739 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:53,739 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. started... 2023-03-14 21:53:53,739 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 664facdeb00d26e025c40b36746b659d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:53:54,160 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209f6ccd118d0b4014a9c6621cd73bc3c0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511209f6ccd118d0b4014a9c6621cd73bc3c0 2023-03-14 21:53:54,169 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/3951f81a0297454f950d59649e00fd32 2023-03-14 21:53:54,177 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/3951f81a0297454f950d59649e00fd32 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32 2023-03-14 21:53:54,185 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32, entries=1, sequenceid=41, filesize=5.0 K 2023-03-14 21:53:54,186 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 664facdeb00d26e025c40b36746b659d in 447ms, sequenceid=41, compaction requested=true 2023-03-14 21:53:54,186 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:54,186 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:54,186 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:53:54,187 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:53:54,187 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,187 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-14 21:53:54,187 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-14 21:53:54,196 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,196 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:54,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:54,196 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,196 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:53:54,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:54,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:54,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:54,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-14 21:53:54,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31d95ff9[Count = 0] remaining members to acquire global barrier 2023-03-14 21:53:54,198 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-14 21:53:54,198 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,199 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,200 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-14 21:53:54,200 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-14 21:53:54,200 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,200 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:53:54,200 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:53:54,206 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-14 21:53:54,206 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,206 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:54,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:54,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:54,206 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-14 21:53:54,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:54,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:54,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:54,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:53:54,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-14 21:53:54,210 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-14 21:53:54,211 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:53:54,211 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:53:54,211 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,211 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:53:54,212 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,213 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,213 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:53:54,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:54,213 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:53:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:53:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,215 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:53:54,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,217 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,217 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:53:54,217 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:53:54,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-14 21:53:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:53:54,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:53:54,217 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:53:54,217 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,217 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:53:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-14 21:53:54,218 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:53:54,318 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:53:54,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:53:54,326 DEBUG [Listener at localhost/45975] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-14 21:53:54,328 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47230, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-14 21:53:54,330 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:54,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:54,343 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:54,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:53:54,344 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831434344 2023-03-14 21:53:54,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:53:54,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-14 21:53:54,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:54,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x74696022 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:54,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78bca973, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:54,381 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,255 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b4049b77706c437f9f60cc6ef95d3306 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103b4049b77706c437f9f60cc6ef95d3306 2023-03-14 21:53:55,256 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:55,274 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:53:55,276 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59120, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:53:55,286 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:53:55,288 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47234, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:53:55,327 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/87226b2effa446e0a77d11a5464fce35 first=Optional[r3] last=Optional[r6] 2023-03-14 21:53:55,341 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/87226b2effa446e0a77d11a5464fce35}] 2023-03-14 21:53:55,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/87226b2effa446e0a77d11a5464fce35 for inclusion in 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:55,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-14 21:53:55,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:53:55,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:55,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/87226b2effa446e0a77d11a5464fce35 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o877gts0smjahngfo31mr7njbnlf7oivgp9g89os2u4rrh7f190a5ri8bgepke70/family1/87226b2effa446e0a77d11a5464fce35 2023-03-14 21:53:55,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o877gts0smjahngfo31mr7njbnlf7oivgp9g89os2u4rrh7f190a5ri8bgepke70/family1/87226b2effa446e0a77d11a5464fce35 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_ 2023-03-14 21:53:55,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o877gts0smjahngfo31mr7njbnlf7oivgp9g89os2u4rrh7f190a5ri8bgepke70/family1/87226b2effa446e0a77d11a5464fce35 into 664facdeb00d26e025c40b36746b659d/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_ - updating store file list. 2023-03-14 21:53:55,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_ into 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:55,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o877gts0smjahngfo31mr7njbnlf7oivgp9g89os2u4rrh7f190a5ri8bgepke70/family1/87226b2effa446e0a77d11a5464fce35 into 664facdeb00d26e025c40b36746b659d/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_) 2023-03-14 21:53:55,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o877gts0smjahngfo31mr7njbnlf7oivgp9g89os2u4rrh7f190a5ri8bgepke70/family1/87226b2effa446e0a77d11a5464fce35 2023-03-14 21:53:55,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:53:55,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:53:55,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 664facdeb00d26e025c40b36746b659d/family1 is initiating minor compaction 2023-03-14 21:53:55,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5af6148; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:53:55,387 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 664facdeb00d26e025c40b36746b659d/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:55,388 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp, totalSize=50.2 K 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting cb0f1cb83ce44c4097add2c84a33c777, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting b8f23bd84b004806a670602ff9bda5fc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 4d1a9ad85008471ebc17f36c24e4de58, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting ff0e8d69bd1946a7900263a89c6ab277, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 506f5c566d2e47d888cfbc652fe977d8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 9d65a823c22b4aa881d627ce14183115, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6141b73e966f4d3b85f9915c653010d1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d37ff076c52045fa8be3e963cb796453, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 3951f81a0297454f950d59649e00fd32, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:53:55,389 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:53:55,402 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:55,403 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:53:55,409 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf8da4f9c3294930b212c91cd843b88e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf8da4f9c3294930b212c91cd843b88e 2023-03-14 21:53:55,410 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231c32bb9f5e99048d4a6ee76ca643a3dd9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231c32bb9f5e99048d4a6ee76ca643a3dd9 2023-03-14 21:53:55,412 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010179a08fe17ecd41d99f14fe66c7305122 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010179a08fe17ecd41d99f14fe66c7305122 2023-03-14 21:53:55,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039815af23549f4d8493fe294aa1bfc01f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601039815af23549f4d8493fe294aa1bfc01f 2023-03-14 21:53:55,414 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-14 21:53:55,414 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:55,444 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 664facdeb00d26e025c40b36746b659d#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:53:55,472 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/da4c147953c34e9484e1baa68f51c62c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/da4c147953c34e9484e1baa68f51c62c 2023-03-14 21:53:55,493 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 664facdeb00d26e025c40b36746b659d/family1 of 664facdeb00d26e025c40b36746b659d into da4c147953c34e9484e1baa68f51c62c(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:53:55,493 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:55,494 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678830835385; duration=0sec 2023-03-14 21:53:55,495 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:53:55,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_] to archive 2023-03-14 21:53:55,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:53:55,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/cb0f1cb83ce44c4097add2c84a33c777 2023-03-14 21:53:55,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/b8f23bd84b004806a670602ff9bda5fc 2023-03-14 21:53:55,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/4d1a9ad85008471ebc17f36c24e4de58 2023-03-14 21:53:55,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/ff0e8d69bd1946a7900263a89c6ab277 2023-03-14 21:53:55,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/506f5c566d2e47d888cfbc652fe977d8 2023-03-14 21:53:55,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/9d65a823c22b4aa881d627ce14183115 2023-03-14 21:53:55,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/6141b73e966f4d3b85f9915c653010d1 2023-03-14 21:53:55,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/d37ff076c52045fa8be3e963cb796453 2023-03-14 21:53:55,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/3951f81a0297454f950d59649e00fd32 2023-03-14 21:53:55,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/082f3a942fe54bd4bd885ecd3f788b76_SeqId_44_ 2023-03-14 21:53:55,916 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205cbbff00ce40543cebe908af97765b1e7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205cbbff00ce40543cebe908af97765b1e7 2023-03-14 21:53:55,918 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:55,957 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/625b97d02ad94e479f2c07cb9988c7d2 first=Optional[r00] last=Optional[r2] 2023-03-14 21:53:55,958 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/625b97d02ad94e479f2c07cb9988c7d2}] 2023-03-14 21:53:55,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/625b97d02ad94e479f2c07cb9988c7d2 for inclusion in 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:55,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-14 21:53:55,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:53:55,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:55,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/625b97d02ad94e479f2c07cb9988c7d2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7ar6grt1jl8dul6vmfj8finh2j241okgtd52ott0gmd0e42rbbam2gbtb5bafh90/family1/625b97d02ad94e479f2c07cb9988c7d2 2023-03-14 21:53:55,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7ar6grt1jl8dul6vmfj8finh2j241okgtd52ott0gmd0e42rbbam2gbtb5bafh90/family1/625b97d02ad94e479f2c07cb9988c7d2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_ 2023-03-14 21:53:55,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7ar6grt1jl8dul6vmfj8finh2j241okgtd52ott0gmd0e42rbbam2gbtb5bafh90/family1/625b97d02ad94e479f2c07cb9988c7d2 into 664facdeb00d26e025c40b36746b659d/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_ - updating store file list. 2023-03-14 21:53:55,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_ into 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:55,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7ar6grt1jl8dul6vmfj8finh2j241okgtd52ott0gmd0e42rbbam2gbtb5bafh90/family1/625b97d02ad94e479f2c07cb9988c7d2 into 664facdeb00d26e025c40b36746b659d/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_) 2023-03-14 21:53:55,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7ar6grt1jl8dul6vmfj8finh2j241okgtd52ott0gmd0e42rbbam2gbtb5bafh90/family1/625b97d02ad94e479f2c07cb9988c7d2 2023-03-14 21:53:55,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:53:55,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:53:55,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 664facdeb00d26e025c40b36746b659d/family1 is initiating minor compaction (all files) 2023-03-14 21:53:55,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@71fb8bdf; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:53:55,990 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 664facdeb00d26e025c40b36746b659d/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:55,991 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/da4c147953c34e9484e1baa68f51c62c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp, totalSize=358.6 K 2023-03-14 21:53:55,991 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 39407c48d187460c96c11675fedaea8f, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:53:55,992 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting da4c147953c34e9484e1baa68f51c62c, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:53:55,993 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 108396655ed745cb824c67ed11661bd5_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845591 2023-03-14 21:53:55,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:55,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:53:55,998 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307b7e55e16ec748bcb9e27a498f8d0d63 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511307b7e55e16ec748bcb9e27a498f8d0d63 2023-03-14 21:53:56,001 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f4e87121b9754de0bca5e9afd0e07613 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f4e87121b9754de0bca5e9afd0e07613 2023-03-14 21:53:56,003 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055eb5ac9008304f3a9e937bbb60cf93d1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055eb5ac9008304f3a9e937bbb60cf93d1 2023-03-14 21:53:56,003 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-14 21:53:56,003 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-14 21:53:56,005 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:56,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:56,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:53:56,007 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:56,013 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6720 sec 2023-03-14 21:53:56,050 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 664facdeb00d26e025c40b36746b659d#family1#compaction#11 average throughput is 5.16 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:53:56,158 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/.tmp/family1/fb738863561345f19055201bcd5796d1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/fb738863561345f19055201bcd5796d1 2023-03-14 21:53:56,169 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 664facdeb00d26e025c40b36746b659d/family1 of 664facdeb00d26e025c40b36746b659d into fb738863561345f19055201bcd5796d1(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:53:56,169 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:56,170 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678830835989; duration=0sec 2023-03-14 21:53:56,170 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:53:56,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/da4c147953c34e9484e1baa68f51c62c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_] to archive 2023-03-14 21:53:56,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:53:56,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/39407c48d187460c96c11675fedaea8f 2023-03-14 21:53:56,228 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/da4c147953c34e9484e1baa68f51c62c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/da4c147953c34e9484e1baa68f51c62c 2023-03-14 21:53:56,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/108396655ed745cb824c67ed11661bd5_SeqId_47_ 2023-03-14 21:53:56,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-14 21:53:56,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:56,690 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-14 21:53:56,692 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:56,693 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-14 21:53:56,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-14 21:53:56,710 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, REOPEN/MOVE}] 2023-03-14 21:53:56,711 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, REOPEN/MOVE 2023-03-14 21:53:56,712 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:56,712 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830836712"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830836712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830836712"}]},"ts":"1678830836712"} 2023-03-14 21:53:56,715 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:56,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:56,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 664facdeb00d26e025c40b36746b659d, disabling compactions & flushes 2023-03-14 21:53:56,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:56,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:56,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. after waiting 0 ms 2023-03-14 21:53:56,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:56,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-14 21:53:56,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:56,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:56,881 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 664facdeb00d26e025c40b36746b659d to self. 2023-03-14 21:53:56,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:56,884 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=CLOSED 2023-03-14 21:53:56,884 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830836884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830836884"}]},"ts":"1678830836884"} 2023-03-14 21:53:56,888 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-14 21:53:56,888 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197 in 171 msec 2023-03-14 21:53:56,889 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=true 2023-03-14 21:53:57,039 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:57,040 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830837039"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830837039"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830837039"}]},"ts":"1678830837039"} 2023-03-14 21:53:57,042 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:57,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:57,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 664facdeb00d26e025c40b36746b659d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:57,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:57,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,201 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,202 DEBUG [StoreOpener-664facdeb00d26e025c40b36746b659d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:57,202 DEBUG [StoreOpener-664facdeb00d26e025c40b36746b659d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:57,203 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 664facdeb00d26e025c40b36746b659d columnFamilyName family1 2023-03-14 21:53:57,210 DEBUG [StoreOpener-664facdeb00d26e025c40b36746b659d-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/fb738863561345f19055201bcd5796d1 2023-03-14 21:53:57,211 INFO [StoreOpener-664facdeb00d26e025c40b36746b659d-1] regionserver.HStore(310): Store=664facdeb00d26e025c40b36746b659d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:57,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:53:57,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 664facdeb00d26e025c40b36746b659d; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60619495, jitterRate=-0.09669913351535797}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:53:57,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:57,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., pid=17, masterSystemTime=1678830837195 2023-03-14 21:53:57,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:57,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:53:57,222 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=664facdeb00d26e025c40b36746b659d, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:57,222 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678830837222"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830837222"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830837222"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830837222"}]},"ts":"1678830837222"} 2023-03-14 21:53:57,226 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-14 21:53:57,226 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 664facdeb00d26e025c40b36746b659d, server=jenkins-hbase4.apache.org,35143,1678830821197 in 182 msec 2023-03-14 21:53:57,228 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-14 21:53:57,228 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=664facdeb00d26e025c40b36746b659d, REOPEN/MOVE in 516 msec 2023-03-14 21:53:57,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-14 21:53:57,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 533 msec 2023-03-14 21:53:57,233 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 964 msec 2023-03-14 21:53:57,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-14 21:53:57,243 INFO [Listener at localhost/45975] compactions.TestMobCompactor(278): alter status finished 2023-03-14 21:53:57,244 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-14 21:53:57,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:57,246 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:57,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:53:57,248 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831437247 2023-03-14 21:53:57,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:53:57,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-14 21:53:57,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:57,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x5bdcbebd to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:53:57,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@343bc84b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:53:57,258 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:57,258 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:57,258 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:57,258 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:57,258 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:58,096 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289c8d888d56754180b2492771cfa5cf13 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289c8d888d56754180b2492771cfa5cf13 2023-03-14 21:53:58,097 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:58,110 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:53:58,111 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59128, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:53:58,118 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:53:58,120 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:53:58,131 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/16882052a1fd4cbbb856d43ed3ec4bd9 first=Optional[r05] last=Optional[r8] 2023-03-14 21:53:58,131 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/16882052a1fd4cbbb856d43ed3ec4bd9}] 2023-03-14 21:53:58,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/16882052a1fd4cbbb856d43ed3ec4bd9 for inclusion in 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:58,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-14 21:53:58,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:53:58,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:53:58,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/16882052a1fd4cbbb856d43ed3ec4bd9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u10e2mrvop7g19h88ck69fqobu4smq72jnmva2md8724i4pb531vu51mrrhck2i4/family1/16882052a1fd4cbbb856d43ed3ec4bd9 2023-03-14 21:53:58,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u10e2mrvop7g19h88ck69fqobu4smq72jnmva2md8724i4pb531vu51mrrhck2i4/family1/16882052a1fd4cbbb856d43ed3ec4bd9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/533bc24e328d4a05a6c5f3090e24572c_SeqId_53_ 2023-03-14 21:53:58,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u10e2mrvop7g19h88ck69fqobu4smq72jnmva2md8724i4pb531vu51mrrhck2i4/family1/16882052a1fd4cbbb856d43ed3ec4bd9 into 664facdeb00d26e025c40b36746b659d/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/533bc24e328d4a05a6c5f3090e24572c_SeqId_53_ - updating store file list. 2023-03-14 21:53:58,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/533bc24e328d4a05a6c5f3090e24572c_SeqId_53_ into 664facdeb00d26e025c40b36746b659d/family1 2023-03-14 21:53:58,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u10e2mrvop7g19h88ck69fqobu4smq72jnmva2md8724i4pb531vu51mrrhck2i4/family1/16882052a1fd4cbbb856d43ed3ec4bd9 into 664facdeb00d26e025c40b36746b659d/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/family1/533bc24e328d4a05a6c5f3090e24572c_SeqId_53_) 2023-03-14 21:53:58,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__u10e2mrvop7g19h88ck69fqobu4smq72jnmva2md8724i4pb531vu51mrrhck2i4/family1/16882052a1fd4cbbb856d43ed3ec4bd9 2023-03-14 21:53:58,158 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:58,158 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:53:58,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511209f6ccd118d0b4014a9c6621cd73bc3c0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511209f6ccd118d0b4014a9c6621cd73bc3c0 2023-03-14 21:53:58,162 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128d47b06326e11424481339f1296deecb6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128d47b06326e11424481339f1296deecb6 2023-03-14 21:53:58,162 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-14 21:53:58,162 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-14 21:53:58,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:58,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:58,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-14 21:53:58,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-14 21:53:58,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:53:58,163 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-14 21:53:58,165 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 918 msec 2023-03-14 21:53:58,247 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=252 (was 210) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x74696022-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:46098 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4179515d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:35143-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45996 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1d000a65-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45972 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45874 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:46040 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:46052 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45914 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x5bdcbebd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45892 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:46092 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x74696022-SendThread(127.0.0.1:58882) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-106488468_15 at /127.0.0.1:45898 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:47230 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x5bdcbebd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/993429238.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4179515d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45934 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1d000a65-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1d000a65-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1d000a65-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:46038 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4179515d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x5bdcbebd-SendThread(127.0.0.1:58882) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:47190 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45986 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58882@0x74696022 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/993429238.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_614626178_15 at /127.0.0.1:45930 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1d000a65-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=593 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=295 (was 314), ProcessCount=169 (was 169), AvailableMemoryMB=3283 (was 3572) 2023-03-14 21:53:58,260 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=252, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=169, AvailableMemoryMB=3283 2023-03-14 21:53:58,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-14 21:53:58,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-14 21:53:58,265 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:53:58,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-14 21:53:58,265 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:53:58,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:53:58,267 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:58,268 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7 empty. 2023-03-14 21:53:58,268 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:58,268 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-14 21:53:58,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:53:58,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:53:58,678 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-14 21:53:58,680 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1720ef3acd9355f39042767aa58726e7, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:53:58,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:53:59,086 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:59,086 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 1720ef3acd9355f39042767aa58726e7, disabling compactions & flushes 2023-03-14 21:53:59,086 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,087 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,087 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. after waiting 0 ms 2023-03-14 21:53:59,087 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,087 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,087 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:53:59,090 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:53:59,091 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830839091"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830839091"}]},"ts":"1678830839091"} 2023-03-14 21:53:59,093 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:53:59,094 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:53:59,094 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830839094"}]},"ts":"1678830839094"} 2023-03-14 21:53:59,096 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-14 21:53:59,194 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1720ef3acd9355f39042767aa58726e7, ASSIGN}] 2023-03-14 21:53:59,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1720ef3acd9355f39042767aa58726e7, ASSIGN 2023-03-14 21:53:59,197 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1720ef3acd9355f39042767aa58726e7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:53:59,348 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1720ef3acd9355f39042767aa58726e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:59,348 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830839348"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830839348"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830839348"}]},"ts":"1678830839348"} 2023-03-14 21:53:59,350 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 1720ef3acd9355f39042767aa58726e7, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:53:59,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:53:59,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1720ef3acd9355f39042767aa58726e7, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:53:59,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:53:59,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,509 INFO [StoreOpener-1720ef3acd9355f39042767aa58726e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,511 DEBUG [StoreOpener-1720ef3acd9355f39042767aa58726e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:53:59,511 DEBUG [StoreOpener-1720ef3acd9355f39042767aa58726e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:53:59,512 INFO [StoreOpener-1720ef3acd9355f39042767aa58726e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1720ef3acd9355f39042767aa58726e7 columnFamilyName family1 2023-03-14 21:53:59,513 INFO [StoreOpener-1720ef3acd9355f39042767aa58726e7-1] regionserver.HStore(310): Store=1720ef3acd9355f39042767aa58726e7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:53:59,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:53:59,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:53:59,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1720ef3acd9355f39042767aa58726e7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59706942, jitterRate=-0.11029723286628723}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:53:59,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:53:59,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., pid=21, masterSystemTime=1678830839503 2023-03-14 21:53:59,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:53:59,526 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1720ef3acd9355f39042767aa58726e7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:53:59,526 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830839526"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830839526"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830839526"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830839526"}]},"ts":"1678830839526"} 2023-03-14 21:53:59,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-14 21:53:59,530 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 1720ef3acd9355f39042767aa58726e7, server=jenkins-hbase4.apache.org,35143,1678830821197 in 178 msec 2023-03-14 21:53:59,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-14 21:53:59,533 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=1720ef3acd9355f39042767aa58726e7, ASSIGN in 336 msec 2023-03-14 21:53:59,533 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:53:59,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830839533"}]},"ts":"1678830839533"} 2023-03-14 21:53:59,535 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-14 21:53:59,563 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:53:59,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.3020 sec 2023-03-14 21:54:00,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-14 21:54:00,372 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-14 21:54:00,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:00,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,406 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:00,406 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:00,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:00,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:00,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,447 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,447 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:00,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:00,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:00,448 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,448 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:00,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:00,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,449 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:00,449 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:00,450 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:00,450 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:00,450 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:00,450 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:00,450 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:00,451 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:54:00,487 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130910854ad5076409eaed2f1cf57f9a77f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130910854ad5076409eaed2f1cf57f9a77f 2023-03-14 21:54:00,878 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:00,891 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/c9659ab8df574bd18c45a2fd56902859 2023-03-14 21:54:00,898 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/c9659ab8df574bd18c45a2fd56902859 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859 2023-03-14 21:54:00,905 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-14 21:54:00,906 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 456ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:00,907 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:00,907 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:00,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:00,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:00,910 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,911 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:00,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:00,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:00,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:00,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16b6fd56[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:00,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:00,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,914 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,914 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:00,915 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,915 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:00,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:00,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:00,917 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,917 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:00,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:00,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:00,917 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:00,918 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:00,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:00,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:00,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:00,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:00,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:00,921 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:00,921 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:00,922 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:00,922 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,922 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:00,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:00,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:00,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:00,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:00,926 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,927 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:00,931 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:00,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:00,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:00,931 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:00,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:00,932 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:00,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:00,932 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,932 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:00,932 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:00,932 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:00,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:00,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:01,032 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:01,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:01,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:01,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:01,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,040 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:01,040 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:01,041 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:01,041 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:01,041 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,041 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,043 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,043 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:01,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:01,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:01,043 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,043 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:01,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,045 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:01,045 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:01,045 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:01,045 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:01,045 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:01,045 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:01,045 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:01,045 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:54:01,460 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112829e526f03ab640f4ab55b6bdd6dc0082 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112829e526f03ab640f4ab55b6bdd6dc0082 2023-03-14 21:54:01,867 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2 2023-03-14 21:54:01,874 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2 2023-03-14 21:54:01,881 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:01,882 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 837ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:01,882 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:01,882 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:01,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:01,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:01,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:01,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:01,885 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,885 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:01,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:01,885 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,886 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a846ea0[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:01,887 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:01,887 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,889 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:01,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:01,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:01,889 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,890 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:01,891 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:01,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:01,894 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:01,894 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:01,895 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:01,895 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,895 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:01,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:01,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:01,897 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:01,907 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:01,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:01,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:01,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:01,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:01,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:01,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:01,909 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,009 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:02,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:02,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:02,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:02,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,018 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:02,018 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:02,019 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:02,019 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:02,019 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,019 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,021 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,021 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:02,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:02,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:02,021 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,021 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:02,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:02,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:02,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:02,023 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:02,023 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:02,024 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:02,420 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:54:02,441 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120132b18f2795354ee5835a9a53cf4b8518 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132b18f2795354ee5835a9a53cf4b8518 2023-03-14 21:54:02,848 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/5186123072cf4992b5b970a61ff28441 2023-03-14 21:54:02,854 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/5186123072cf4992b5b970a61ff28441 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441 2023-03-14 21:54:02,859 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441, entries=1, sequenceid=13, filesize=4.9 K 2023-03-14 21:54:02,860 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 836ms, sequenceid=13, compaction requested=true 2023-03-14 21:54:02,860 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:02,860 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:02,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:02,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:02,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:02,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:02,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,863 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:02,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:02,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c261baa[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:02,866 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:02,866 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,867 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,868 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:02,868 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:02,868 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,868 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:02,868 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:02,869 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:02,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:02,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:02,870 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:02,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:02,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:02,873 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:02,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:02,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:02,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,873 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:02,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:02,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:02,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,876 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:02,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,877 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:02,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:02,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:02,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:02,881 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:02,882 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:02,882 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:02,882 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,882 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:02,882 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:02,982 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:02,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:02,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:02,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:02,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,990 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:02,991 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:02,991 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:02,991 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:02,992 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,992 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,993 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,993 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:02,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:02,993 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:02,993 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,994 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:02,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:02,995 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:02,995 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:02,995 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:02,995 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:02,995 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:02,995 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:03,011 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f4dfc544eb3f4ad987800e67016362c4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4dfc544eb3f4ad987800e67016362c4 2023-03-14 21:54:03,417 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/3d43c623ec2c47678e67a97347fe7657 2023-03-14 21:54:03,423 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/3d43c623ec2c47678e67a97347fe7657 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657 2023-03-14 21:54:03,428 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657, entries=1, sequenceid=17, filesize=4.9 K 2023-03-14 21:54:03,429 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 434ms, sequenceid=17, compaction requested=true 2023-03-14 21:54:03,429 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:03,429 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:03,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:03,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:03,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:03,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:03,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,431 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:03,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:03,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:03,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@477bbd40[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:03,433 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:03,433 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,434 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,434 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:03,434 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:03,434 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,434 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:03,434 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:03,437 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:03,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,437 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:03,437 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:03,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:03,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:03,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:03,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:03,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:03,439 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:03,439 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:03,440 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:03,440 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,440 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:03,441 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,441 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,441 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:03,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:03,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:03,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:03,442 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:03,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,443 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:03,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:03,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:03,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:03,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:03,446 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:03,446 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:03,445 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,446 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:03,446 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:03,446 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,446 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:03,447 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,447 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:03,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:03,547 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:03,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:03,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:03,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:03,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,555 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:03,555 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:03,555 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:03,555 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:03,556 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,556 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,557 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,557 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:03,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:03,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:03,558 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:03,558 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:03,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,559 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:03,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:03,559 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:03,559 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:03,560 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:03,560 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:03,560 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:03,560 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:03,560 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:03,560 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:03,978 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286d47679f18ec4c36b154a6b77f1835cb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512286d47679f18ec4c36b154a6b77f1835cb 2023-03-14 21:54:04,384 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/80957ec8a5e94d25b3b58a15d68a2dc3 2023-03-14 21:54:04,390 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/80957ec8a5e94d25b3b58a15d68a2dc3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3 2023-03-14 21:54:04,396 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3, entries=1, sequenceid=21, filesize=4.9 K 2023-03-14 21:54:04,397 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 837ms, sequenceid=21, compaction requested=true 2023-03-14 21:54:04,397 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:04,398 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:04,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:04,401 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:04,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:04,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:04,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6686b4b4[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:04,403 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:04,403 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,404 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,404 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:04,404 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:04,404 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,404 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:04,404 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:04,406 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:04,406 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,406 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,406 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:04,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:04,410 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:04,410 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:04,410 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:04,410 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,410 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:04,415 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,415 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,416 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:04,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,416 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:04,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,417 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:04,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,419 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,419 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,419 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:04,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:04,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:04,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:04,420 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:04,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:04,421 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:04,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,421 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:04,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:04,421 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,421 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,521 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:04,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:04,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:04,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:04,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,529 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:04,529 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:04,529 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:04,529 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:04,530 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,530 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,531 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,531 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:04,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:04,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:04,531 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:04,532 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:04,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:04,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:04,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:04,533 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:04,533 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:04,533 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:04,533 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:04,533 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:04,533 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:04,533 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:04,534 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:04,950 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231df03012770154bbdb71e4348967f0d23 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231df03012770154bbdb71e4348967f0d23 2023-03-14 21:54:05,358 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/93e7eb6bf29f44c988ba988a835a8b4d 2023-03-14 21:54:05,366 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/93e7eb6bf29f44c988ba988a835a8b4d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d 2023-03-14 21:54:05,372 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d, entries=1, sequenceid=25, filesize=4.9 K 2023-03-14 21:54:05,373 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 839ms, sequenceid=25, compaction requested=true 2023-03-14 21:54:05,373 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:05,373 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:05,376 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,376 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,377 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,377 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fae27aa[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:05,379 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:05,379 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,380 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,380 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:05,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:05,382 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:05,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:05,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:05,385 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:05,385 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:05,385 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:05,386 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,386 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:05,387 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,387 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,387 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,388 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,389 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,396 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:05,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:05,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:05,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:05,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:05,397 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,397 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,397 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,397 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:05,397 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:05,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:05,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:05,498 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:05,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:05,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:05,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:05,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,506 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:05,506 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:05,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:05,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:05,508 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,508 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,509 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,509 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:05,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:05,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:05,509 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,509 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:05,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:05,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:05,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,510 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:05,510 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:05,511 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:05,511 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:05,511 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:05,511 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:05,934 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f43fdf0fc5b148aaaf12466eda406756 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101f43fdf0fc5b148aaaf12466eda406756 2023-03-14 21:54:05,944 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/73ea5c08927e465ebde7232e2e70c72e 2023-03-14 21:54:05,953 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/73ea5c08927e465ebde7232e2e70c72e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e 2023-03-14 21:54:05,960 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e, entries=1, sequenceid=29, filesize=4.9 K 2023-03-14 21:54:05,961 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 450ms, sequenceid=29, compaction requested=true 2023-03-14 21:54:05,961 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:05,961 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:05,961 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:05,961 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:05,961 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,961 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:05,961 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:05,963 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,963 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,964 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,964 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33d980be[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:05,966 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:05,966 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,967 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,968 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:05,968 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:05,968 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:05,968 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,968 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:05,969 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,970 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:05,970 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:05,970 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:05,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:05,976 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:05,976 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:05,976 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:05,976 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,976 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:05,979 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,979 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:05,979 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:05,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,980 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:05,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:05,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:05,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,983 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,983 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:05,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:05,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:05,984 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:05,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:05,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:05,984 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:05,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:05,984 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:05,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:06,085 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:06,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:06,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:06,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:06,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,097 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:06,097 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:06,098 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:06,098 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:06,098 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,098 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,100 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,100 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:06,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:06,100 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,100 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,101 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:06,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,102 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:06,102 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:06,102 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:06,102 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:06,103 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:06,103 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:06,103 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:06,103 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:06,521 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103eef821b295614461abbe857ce018572b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103eef821b295614461abbe857ce018572b 2023-03-14 21:54:06,927 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/d58a867c38c84cbf926b6e8f657e953b 2023-03-14 21:54:06,934 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/d58a867c38c84cbf926b6e8f657e953b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b 2023-03-14 21:54:06,939 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b, entries=1, sequenceid=33, filesize=4.9 K 2023-03-14 21:54:06,940 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 837ms, sequenceid=33, compaction requested=true 2023-03-14 21:54:06,940 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:06,940 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:06,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:06,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:06,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,941 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:06,941 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:06,942 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,942 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:06,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:06,943 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,943 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:06,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:06,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:06,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:06,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:06,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2952aa38[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:06,944 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:06,944 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,946 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:06,946 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:06,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:06,955 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,955 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:06,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:06,958 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:06,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:06,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:06,958 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,958 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:06,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,960 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:06,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:06,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:06,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:06,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:06,961 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:06,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:06,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,962 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:06,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,964 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,964 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:06,964 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:06,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:06,964 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:06,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:06,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:06,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:06,964 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:06,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:06,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:06,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,066 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:07,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:07,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:07,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:07,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,074 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:07,074 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:07,075 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:07,075 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:07,075 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,075 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,076 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,076 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:07,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:07,077 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,077 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:07,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:07,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:07,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:07,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:07,078 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:07,079 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:07,079 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:07,079 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:07,079 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:07,500 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070061158a837c42ecacd9a6d6b1dcf661 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509070061158a837c42ecacd9a6d6b1dcf661 2023-03-14 21:54:07,906 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/9c28d5869ae9414194850a0ae0731c27 2023-03-14 21:54:07,912 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/9c28d5869ae9414194850a0ae0731c27 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27 2023-03-14 21:54:07,917 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27, entries=1, sequenceid=37, filesize=4.9 K 2023-03-14 21:54:07,918 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 839ms, sequenceid=37, compaction requested=true 2023-03-14 21:54:07,918 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:07,918 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:07,919 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:07,919 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:07,919 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,919 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:07,919 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:07,921 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,921 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:07,921 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,921 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:07,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:07,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:07,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a2298b2[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:07,923 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:07,923 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,924 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:07,924 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:07,924 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,924 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:07,924 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:07,925 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:07,925 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,925 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:07,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,926 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:07,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:07,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:07,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:07,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:07,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:07,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:07,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:07,928 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:07,928 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:07,928 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:07,928 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,928 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:07,930 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,930 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,930 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:07,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:07,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:07,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:07,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:07,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:07,931 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:07,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,932 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:07,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:07,936 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:07,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:07,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:07,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:07,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:07,936 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:07,937 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,937 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,937 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:07,937 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:07,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:07,937 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:08,037 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:08,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:08,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:08,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:08,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,045 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:08,045 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:08,046 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:54:08,046 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:08,046 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,046 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,047 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,047 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:08,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:08,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:08,048 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,048 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:08,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:08,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:54:08,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,049 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:54:08,049 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:08,050 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:54:08,050 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:08,050 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:08,050 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:08,050 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. started... 2023-03-14 21:54:08,050 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1720ef3acd9355f39042767aa58726e7 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:08,465 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511207489e31b987d4f0d8ba077f24b9d12eb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511207489e31b987d4f0d8ba077f24b9d12eb 2023-03-14 21:54:08,871 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/c733523dc5f143e9a7432039578b3694 2023-03-14 21:54:08,877 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/c733523dc5f143e9a7432039578b3694 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694 2023-03-14 21:54:08,883 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694, entries=1, sequenceid=41, filesize=4.9 K 2023-03-14 21:54:08,884 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1720ef3acd9355f39042767aa58726e7 in 834ms, sequenceid=41, compaction requested=true 2023-03-14 21:54:08,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:08,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:08,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:08,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:08,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:54:08,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:54:08,886 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:08,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:08,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:54:08,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@347f732b[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:08,887 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:08,888 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,889 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:08,889 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,889 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:08,892 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:08,895 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:54:08,895 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:08,895 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:08,895 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,895 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:08,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:08,896 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,897 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:08,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:08,900 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:08,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:08,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:08,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:54:08,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-14 21:54:08,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:08,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:08,901 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:08,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:09,002 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:09,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:09,004 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-14 21:54:09,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:09,007 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:09,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:09,008 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831449008 2023-03-14 21:54:09,009 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:09,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-14 21:54:09,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:09,011 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x2f08c329 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:09,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@459b2f13, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:09,016 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:09,450 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033e54acebbd0a4d9a9f6887e429f791a3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601033e54acebbd0a4d9a9f6887e429f791a3 2023-03-14 21:54:09,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:09,464 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:09,465 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:09,490 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:09,490 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:09,490 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5bdcbebd to 127.0.0.1:58882 2023-03-14 21:54:09,490 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:09,491 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33616, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:09,492 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:09,492 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74696022 to 127.0.0.1:58882 2023-03-14 21:54:09,492 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:09,503 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/403b88eab52d438a843f697fee701738 first=Optional[r5] last=Optional[r6] 2023-03-14 21:54:09,504 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/403b88eab52d438a843f697fee701738}] 2023-03-14 21:54:09,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/403b88eab52d438a843f697fee701738 for inclusion in 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:09,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-14 21:54:09,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:09,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:09,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/403b88eab52d438a843f697fee701738 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__dgst4iag5b41ov7jovjqj8vs3p277tvvirfvtucj8l01fbkoibri8g3qeh8lnd3d/family1/403b88eab52d438a843f697fee701738 2023-03-14 21:54:09,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__dgst4iag5b41ov7jovjqj8vs3p277tvvirfvtucj8l01fbkoibri8g3qeh8lnd3d/family1/403b88eab52d438a843f697fee701738 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_ 2023-03-14 21:54:09,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__dgst4iag5b41ov7jovjqj8vs3p277tvvirfvtucj8l01fbkoibri8g3qeh8lnd3d/family1/403b88eab52d438a843f697fee701738 into 1720ef3acd9355f39042767aa58726e7/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_ - updating store file list. 2023-03-14 21:54:09,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_ into 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:09,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__dgst4iag5b41ov7jovjqj8vs3p277tvvirfvtucj8l01fbkoibri8g3qeh8lnd3d/family1/403b88eab52d438a843f697fee701738 into 1720ef3acd9355f39042767aa58726e7/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_) 2023-03-14 21:54:09,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__dgst4iag5b41ov7jovjqj8vs3p277tvvirfvtucj8l01fbkoibri8g3qeh8lnd3d/family1/403b88eab52d438a843f697fee701738 2023-03-14 21:54:09,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:54:09,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:54:09,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 1720ef3acd9355f39042767aa58726e7/family1 is initiating minor compaction 2023-03-14 21:54:09,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@540101fd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:09,524 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1720ef3acd9355f39042767aa58726e7/family1 in testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:09,524 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp, totalSize=49.7 K 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6fb0f8dfc47f405d8235aa85b6bb3ff2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5186123072cf4992b5b970a61ff28441, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 3d43c623ec2c47678e67a97347fe7657, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 80957ec8a5e94d25b3b58a15d68a2dc3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 93e7eb6bf29f44c988ba988a835a8b4d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 73ea5c08927e465ebde7232e2e70c72e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d58a867c38c84cbf926b6e8f657e953b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 9c28d5869ae9414194850a0ae0731c27, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c733523dc5f143e9a7432039578b3694, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:54:09,525 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 296b6caca02c44059aef0a6950c199a9_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:54:09,526 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:09,527 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:09,529 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101f43fdf0fc5b148aaaf12466eda406756 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101f43fdf0fc5b148aaaf12466eda406756 2023-03-14 21:54:09,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103eef821b295614461abbe857ce018572b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103eef821b295614461abbe857ce018572b 2023-03-14 21:54:09,530 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-14 21:54:09,530 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-14 21:54:09,531 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:09,531 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:09,580 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1720ef3acd9355f39042767aa58726e7#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:10,401 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f17c0a2d0c7c48908df0b60f3ed8ee00 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231f17c0a2d0c7c48908df0b60f3ed8ee00 2023-03-14 21:54:10,402 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/666c0b35a5544ebbb5d8c5d1455472cc as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/666c0b35a5544ebbb5d8c5d1455472cc 2023-03-14 21:54:10,402 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:10,409 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1720ef3acd9355f39042767aa58726e7/family1 of 1720ef3acd9355f39042767aa58726e7 into 666c0b35a5544ebbb5d8c5d1455472cc(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:54:10,409 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:10,409 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678830849524; duration=0sec 2023-03-14 21:54:10,409 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:10,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_] to archive 2023-03-14 21:54:10,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:10,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/6fb0f8dfc47f405d8235aa85b6bb3ff2 2023-03-14 21:54:10,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/5186123072cf4992b5b970a61ff28441 2023-03-14 21:54:10,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3d43c623ec2c47678e67a97347fe7657 2023-03-14 21:54:10,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/80957ec8a5e94d25b3b58a15d68a2dc3 2023-03-14 21:54:10,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/93e7eb6bf29f44c988ba988a835a8b4d 2023-03-14 21:54:10,438 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9a7edf9f5bde4cbf83b663ed06efafbe first=Optional[r1] last=Optional[r4] 2023-03-14 21:54:10,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/73ea5c08927e465ebde7232e2e70c72e 2023-03-14 21:54:10,438 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9a7edf9f5bde4cbf83b663ed06efafbe}] 2023-03-14 21:54:10,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/d58a867c38c84cbf926b6e8f657e953b 2023-03-14 21:54:10,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9a7edf9f5bde4cbf83b663ed06efafbe for inclusion in 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:10,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9c28d5869ae9414194850a0ae0731c27 2023-03-14 21:54:10,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c733523dc5f143e9a7432039578b3694 2023-03-14 21:54:10,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-14 21:54:10,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:10,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:10,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/296b6caca02c44059aef0a6950c199a9_SeqId_44_ 2023-03-14 21:54:10,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9a7edf9f5bde4cbf83b663ed06efafbe to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__v5aiji7bhllaehpmk001kbafvao28ujvoajb8vu6kmcl7lidp18hurl1vg9ven7o/family1/9a7edf9f5bde4cbf83b663ed06efafbe 2023-03-14 21:54:10,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__v5aiji7bhllaehpmk001kbafvao28ujvoajb8vu6kmcl7lidp18hurl1vg9ven7o/family1/9a7edf9f5bde4cbf83b663ed06efafbe as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_ 2023-03-14 21:54:10,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__v5aiji7bhllaehpmk001kbafvao28ujvoajb8vu6kmcl7lidp18hurl1vg9ven7o/family1/9a7edf9f5bde4cbf83b663ed06efafbe into 1720ef3acd9355f39042767aa58726e7/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_ - updating store file list. 2023-03-14 21:54:10,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_ into 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:10,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__v5aiji7bhllaehpmk001kbafvao28ujvoajb8vu6kmcl7lidp18hurl1vg9ven7o/family1/9a7edf9f5bde4cbf83b663ed06efafbe into 1720ef3acd9355f39042767aa58726e7/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_) 2023-03-14 21:54:10,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__v5aiji7bhllaehpmk001kbafvao28ujvoajb8vu6kmcl7lidp18hurl1vg9ven7o/family1/9a7edf9f5bde4cbf83b663ed06efafbe 2023-03-14 21:54:10,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:54:10,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:54:10,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 1720ef3acd9355f39042767aa58726e7/family1 is initiating minor compaction (all files) 2023-03-14 21:54:10,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@13e0cd77; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:10,455 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1720ef3acd9355f39042767aa58726e7/family1 in testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:54:10,455 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/666c0b35a5544ebbb5d8c5d1455472cc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp, totalSize=178.1 K 2023-03-14 21:54:10,456 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c9659ab8df574bd18c45a2fd56902859, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:54:10,456 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 666c0b35a5544ebbb5d8c5d1455472cc, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:54:10,457 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 742d20d36c1f4b57a691cbe6ae547967_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245591 2023-03-14 21:54:10,457 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:10,458 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:10,459 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132b18f2795354ee5835a9a53cf4b8518 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132b18f2795354ee5835a9a53cf4b8518 2023-03-14 21:54:10,461 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4dfc544eb3f4ad987800e67016362c4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4dfc544eb3f4ad987800e67016362c4 2023-03-14 21:54:10,462 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512286d47679f18ec4c36b154a6b77f1835cb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512286d47679f18ec4c36b154a6b77f1835cb 2023-03-14 21:54:10,463 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231df03012770154bbdb71e4348967f0d23 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231df03012770154bbdb71e4348967f0d23 2023-03-14 21:54:10,463 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-14 21:54:10,463 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-14 21:54:10,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:10,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:10,476 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1720ef3acd9355f39042767aa58726e7#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:10,531 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/.tmp/family1/9f911de9d11046d2b38242a4da77df5e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/9f911de9d11046d2b38242a4da77df5e 2023-03-14 21:54:10,536 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1720ef3acd9355f39042767aa58726e7/family1 of 1720ef3acd9355f39042767aa58726e7 into 9f911de9d11046d2b38242a4da77df5e(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:54:10,537 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:10,537 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678830850455; duration=0sec 2023-03-14 21:54:10,537 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:10,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/666c0b35a5544ebbb5d8c5d1455472cc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_] to archive 2023-03-14 21:54:10,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:10,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/c9659ab8df574bd18c45a2fd56902859 2023-03-14 21:54:10,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/666c0b35a5544ebbb5d8c5d1455472cc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/666c0b35a5544ebbb5d8c5d1455472cc 2023-03-14 21:54:10,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/742d20d36c1f4b57a691cbe6ae547967_SeqId_47_ 2023-03-14 21:54:11,318 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130062b2ebf56cc40cb99ad9123e36566a4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130062b2ebf56cc40cb99ad9123e36566a4 2023-03-14 21:54:11,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:11,344 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd first=Optional[r00] last=Optional[r8] 2023-03-14 21:54:11,344 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd}] 2023-03-14 21:54:11,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd for inclusion in 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:11,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-14 21:54:11,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:11,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:54:11,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gdolkav1d6od2fcpsq1jhphgilcq58a0bbrfjp2uhu6ttfbknf60q6cepri4b78o/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd 2023-03-14 21:54:11,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gdolkav1d6od2fcpsq1jhphgilcq58a0bbrfjp2uhu6ttfbknf60q6cepri4b78o/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3f932b064a634e6fa81160ca9238fd83_SeqId_50_ 2023-03-14 21:54:11,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gdolkav1d6od2fcpsq1jhphgilcq58a0bbrfjp2uhu6ttfbknf60q6cepri4b78o/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd into 1720ef3acd9355f39042767aa58726e7/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3f932b064a634e6fa81160ca9238fd83_SeqId_50_ - updating store file list. 2023-03-14 21:54:11,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3f932b064a634e6fa81160ca9238fd83_SeqId_50_ into 1720ef3acd9355f39042767aa58726e7/family1 2023-03-14 21:54:11,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gdolkav1d6od2fcpsq1jhphgilcq58a0bbrfjp2uhu6ttfbknf60q6cepri4b78o/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd into 1720ef3acd9355f39042767aa58726e7/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/family1/3f932b064a634e6fa81160ca9238fd83_SeqId_50_) 2023-03-14 21:54:11,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gdolkav1d6od2fcpsq1jhphgilcq58a0bbrfjp2uhu6ttfbknf60q6cepri4b78o/family1/a1c6c3ba39b7492a8b0e6e14e2bcf7bd 2023-03-14 21:54:11,360 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:11,361 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:11,362 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511207489e31b987d4f0d8ba077f24b9d12eb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511207489e31b987d4f0d8ba077f24b9d12eb 2023-03-14 21:54:11,364 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112829e526f03ab640f4ab55b6bdd6dc0082 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112829e526f03ab640f4ab55b6bdd6dc0082 2023-03-14 21:54:11,365 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130910854ad5076409eaed2f1cf57f9a77f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130910854ad5076409eaed2f1cf57f9a77f 2023-03-14 21:54:11,365 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-14 21:54:11,365 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-14 21:54:11,365 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:11,365 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:11,365 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:54:11,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:54:11,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:11,366 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:11,368 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.3610 sec 2023-03-14 21:54:11,470 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249 (was 252), OpenFileDescriptor=593 (was 593), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=280 (was 295), ProcessCount=169 (was 169), AvailableMemoryMB=3769 (was 3283) - AvailableMemoryMB LEAK? - 2023-03-14 21:54:11,483 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=249, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=280, ProcessCount=169, AvailableMemoryMB=3769 2023-03-14 21:54:11,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:54:11,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:11,489 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:54:11,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-14 21:54:11,490 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:54:11,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:11,491 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:11,492 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658 empty. 2023-03-14 21:54:11,493 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:11,493 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-14 21:54:11,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:11,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:11,906 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:11,907 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d56bd8e4f6c9c696d51e6aa54aa60658, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:12,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing d56bd8e4f6c9c696d51e6aa54aa60658, disabling compactions & flushes 2023-03-14 21:54:12,314 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. after waiting 0 ms 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,314 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,314 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:12,316 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:54:12,317 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830852317"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830852317"}]},"ts":"1678830852317"} 2023-03-14 21:54:12,319 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:54:12,320 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:54:12,320 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830852320"}]},"ts":"1678830852320"} 2023-03-14 21:54:12,322 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-14 21:54:12,327 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d56bd8e4f6c9c696d51e6aa54aa60658, ASSIGN}] 2023-03-14 21:54:12,329 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d56bd8e4f6c9c696d51e6aa54aa60658, ASSIGN 2023-03-14 21:54:12,330 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d56bd8e4f6c9c696d51e6aa54aa60658, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:12,481 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=d56bd8e4f6c9c696d51e6aa54aa60658, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:12,481 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830852481"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830852481"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830852481"}]},"ts":"1678830852481"} 2023-03-14 21:54:12,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure d56bd8e4f6c9c696d51e6aa54aa60658, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:12,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:12,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d56bd8e4f6c9c696d51e6aa54aa60658, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:12,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:12,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,639 INFO [StoreOpener-d56bd8e4f6c9c696d51e6aa54aa60658-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,640 DEBUG [StoreOpener-d56bd8e4f6c9c696d51e6aa54aa60658-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:12,640 DEBUG [StoreOpener-d56bd8e4f6c9c696d51e6aa54aa60658-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:12,641 INFO [StoreOpener-d56bd8e4f6c9c696d51e6aa54aa60658-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d56bd8e4f6c9c696d51e6aa54aa60658 columnFamilyName family1 2023-03-14 21:54:12,642 INFO [StoreOpener-d56bd8e4f6c9c696d51e6aa54aa60658-1] regionserver.HStore(310): Store=d56bd8e4f6c9c696d51e6aa54aa60658/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:12,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:54:12,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:12,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d56bd8e4f6c9c696d51e6aa54aa60658; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61403077, jitterRate=-0.08502285182476044}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:12,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:12,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658., pid=25, masterSystemTime=1678830852634 2023-03-14 21:54:12,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:12,651 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=d56bd8e4f6c9c696d51e6aa54aa60658, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:12,651 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830852651"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830852651"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830852651"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830852651"}]},"ts":"1678830852651"} 2023-03-14 21:54:12,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-14 21:54:12,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure d56bd8e4f6c9c696d51e6aa54aa60658, server=jenkins-hbase4.apache.org,35143,1678830821197 in 170 msec 2023-03-14 21:54:12,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-14 21:54:12,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d56bd8e4f6c9c696d51e6aa54aa60658, ASSIGN in 327 msec 2023-03-14 21:54:12,657 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:54:12,657 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830852657"}]},"ts":"1678830852657"} 2023-03-14 21:54:12,658 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-14 21:54:12,661 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:54:12,663 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1750 sec 2023-03-14 21:54:12,880 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:13,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-14 21:54:13,596 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-14 21:54:13,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:13,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,619 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:13,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:13,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:13,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:13,620 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,620 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,621 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:13,621 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:13,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:13,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:13,621 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:13,621 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:13,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:13,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,622 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,622 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:13,622 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:13,622 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:13,622 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:13,623 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:13,623 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:13,623 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:13,623 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:13,624 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:54:14,048 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511303f49aabadbbc4ec4b8a970cd8f14046d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511303f49aabadbbc4ec4b8a970cd8f14046d 2023-03-14 21:54:14,452 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/d2b4c176c9124587915617ab8143c7d2 2023-03-14 21:54:14,458 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/d2b4c176c9124587915617ab8143c7d2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2 2023-03-14 21:54:14,463 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-14 21:54:14,464 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 840ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:14,464 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:14,465 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:14,465 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:14,465 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:14,465 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,465 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:14,465 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:14,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:14,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:14,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:14,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:14,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b5cc96a[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:14,468 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:14,468 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,469 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,469 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:14,469 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:14,469 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:14,469 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,469 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:14,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:14,472 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:14,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:14,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:14,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:14,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:14,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:14,474 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:14,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:14,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:14,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,474 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:14,475 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,475 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:14,475 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:14,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:14,476 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,477 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,479 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,479 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:14,479 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,479 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:14,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:14,479 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:14,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:14,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:14,479 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:14,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:14,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:14,480 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:14,580 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:14,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:14,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:14,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:14,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,587 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:14,587 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:14,587 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:14,587 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:14,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,589 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:14,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:14,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:14,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:14,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:14,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:14,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:14,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:14,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:14,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:14,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:14,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:14,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:14,591 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:14,591 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:14,591 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:54:15,003 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128385a11bdeda24bddbfa87856171ab3e9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128385a11bdeda24bddbfa87856171ab3e9 2023-03-14 21:54:15,409 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/8428c9c7dc354f2f9b81c3d5b4a05789 2023-03-14 21:54:15,415 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/8428c9c7dc354f2f9b81c3d5b4a05789 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789 2023-03-14 21:54:15,419 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:15,420 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 829ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:15,420 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:15,420 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:15,420 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:15,423 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,423 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,423 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,423 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:15,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77e3a953[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:15,425 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:15,425 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,426 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,426 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:15,426 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:15,426 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:15,426 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,427 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:15,428 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:15,428 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,428 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,428 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:15,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:15,430 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:15,430 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:15,430 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:15,430 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,430 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:15,431 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,432 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,432 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,432 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,433 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:15,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:15,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:15,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:15,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,437 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:15,438 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:15,538 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:15,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:15,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:15,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:15,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,545 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:15,545 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:15,545 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:15,545 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:15,545 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,545 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,547 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,547 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:15,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:15,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:15,547 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,547 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:15,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:15,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:15,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:15,548 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:15,548 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:15,548 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:15,962 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201635b4621ee2843629ca3c384e8269853 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201635b4621ee2843629ca3c384e8269853 2023-03-14 21:54:15,967 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/f2682ff14c214fa69428db5040f9d33b 2023-03-14 21:54:15,971 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/f2682ff14c214fa69428db5040f9d33b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b 2023-03-14 21:54:15,975 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b, entries=1, sequenceid=13, filesize=4.9 K 2023-03-14 21:54:15,976 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 428ms, sequenceid=13, compaction requested=true 2023-03-14 21:54:15,976 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:15,976 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:15,976 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:15,976 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:15,976 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,976 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:15,976 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:15,978 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,978 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,979 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,979 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:15,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:15,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@196a5815[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:15,980 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:15,980 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,981 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,981 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:15,981 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:15,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:15,981 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:15,983 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:15,983 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,983 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:15,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,983 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:15,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:15,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:15,985 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:15,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:15,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:15,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,985 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:15,986 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,986 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:15,986 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,988 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,993 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:15,994 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:15,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:15,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:15,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:15,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:15,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:15,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:15,994 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:15,995 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,095 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:16,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:16,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:16,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:16,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,101 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:16,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:16,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:16,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:16,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,103 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,103 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:16,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:16,104 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,104 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:16,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:16,105 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:16,105 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:16,105 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:16,522 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057ac0f9cd825448a38c1772164c02a90b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057ac0f9cd825448a38c1772164c02a90b 2023-03-14 21:54:16,928 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/5be4a460287143abbf72217dbe34b4af 2023-03-14 21:54:16,933 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/5be4a460287143abbf72217dbe34b4af as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af 2023-03-14 21:54:16,937 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af, entries=1, sequenceid=17, filesize=4.9 K 2023-03-14 21:54:16,938 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 833ms, sequenceid=17, compaction requested=true 2023-03-14 21:54:16,938 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:16,938 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:16,940 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:16,940 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:16,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:16,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:16,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56c38638[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:16,941 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:16,941 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,942 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,942 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:16,943 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:16,943 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:16,943 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,943 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:16,945 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:16,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:16,945 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:16,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:16,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:16,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:16,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:16,947 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:16,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:16,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:16,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,947 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:16,948 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,948 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:16,948 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:16,949 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,950 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,951 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,951 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,951 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,951 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:16,952 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:16,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:16,951 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:16,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:16,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:16,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:16,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:16,952 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:17,052 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:17,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:17,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:17,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:17,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,059 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:17,059 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:17,059 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:17,059 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:17,059 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,059 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,061 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,061 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:17,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:17,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:17,061 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,061 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:17,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:17,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:17,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,062 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:17,062 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:17,062 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:17,062 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:17,062 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:17,062 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:17,062 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:17,062 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:17,475 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d61a75d504574a3fbd6072d32aabb536 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d61a75d504574a3fbd6072d32aabb536 2023-03-14 21:54:17,881 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/cc2a344acec14995a607e9c1caeff3ef 2023-03-14 21:54:17,886 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/cc2a344acec14995a607e9c1caeff3ef as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef 2023-03-14 21:54:17,890 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef, entries=1, sequenceid=21, filesize=4.9 K 2023-03-14 21:54:17,891 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 829ms, sequenceid=21, compaction requested=true 2023-03-14 21:54:17,891 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:17,891 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:17,891 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:17,894 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,894 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:17,895 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,895 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b791213[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:17,896 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:17,896 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,897 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,897 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:17,897 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:17,897 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,897 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:17,897 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:17,899 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,899 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,899 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:17,899 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:17,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:17,901 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:17,901 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:17,901 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:17,901 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,901 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:17,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:17,902 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:17,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,903 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:17,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,904 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:17,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:17,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:17,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:17,907 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:17,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,907 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:17,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:17,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:18,007 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:18,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:18,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:18,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:18,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,014 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:18,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:18,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:18,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:18,015 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,015 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,016 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:18,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:18,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:18,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:18,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:18,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,017 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:18,017 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,017 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:18,017 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:18,017 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:18,018 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:18,018 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:18,018 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:18,018 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:18,018 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:18,163 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:54:18,434 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123193e20a7b7c3c46bab4ed69c797218266 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123193e20a7b7c3c46bab4ed69c797218266 2023-03-14 21:54:18,840 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/1ddc1624f0664fba8b19e2f3449ace8f 2023-03-14 21:54:18,847 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/1ddc1624f0664fba8b19e2f3449ace8f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f 2023-03-14 21:54:18,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f, entries=1, sequenceid=25, filesize=4.9 K 2023-03-14 21:54:18,854 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 836ms, sequenceid=25, compaction requested=true 2023-03-14 21:54:18,854 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:18,854 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:18,854 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:18,857 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:18,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:18,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@113a851e[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:18,859 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:18,859 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,861 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:18,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:18,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:18,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:18,866 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:18,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:18,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:18,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:18,869 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:18,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:18,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:18,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,869 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:18,871 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,871 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,871 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:18,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:18,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:18,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:18,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:18,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:18,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,875 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:18,876 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,876 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:18,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:18,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:18,876 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:18,876 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:18,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:18,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:18,976 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:18,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:18,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:18,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:18,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,984 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:18,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:18,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:18,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:18,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:18,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:18,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:18,986 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:18,986 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:18,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:18,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:18,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:18,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:18,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:18,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:18,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:18,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:18,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:18,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:18,987 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:19,402 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010132c479970af1440f8ebbfffd5e0713f5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010132c479970af1440f8ebbfffd5e0713f5 2023-03-14 21:54:19,811 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/425dd4647b124419ba325fcf5e6ff862 2023-03-14 21:54:19,817 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/425dd4647b124419ba325fcf5e6ff862 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862 2023-03-14 21:54:19,825 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862, entries=1, sequenceid=29, filesize=4.9 K 2023-03-14 21:54:19,826 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 839ms, sequenceid=29, compaction requested=true 2023-03-14 21:54:19,826 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:19,826 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:19,826 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:19,827 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:19,827 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,827 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:19,827 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:19,830 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,830 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:19,830 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,830 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:19,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:19,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a5bcddc[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:19,832 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:19,832 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,833 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,833 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:19,833 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:19,833 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:19,833 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,833 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:19,835 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,835 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:19,835 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:19,835 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:19,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:19,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:19,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:19,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:19,838 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:19,838 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:19,838 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:19,838 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,838 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:19,839 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,840 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:19,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,840 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:19,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:19,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,842 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:19,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,845 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,845 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,845 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:19,846 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:19,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:19,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:19,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:19,845 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:19,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:19,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,846 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,946 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:19,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:19,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:19,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:19,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,954 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:19,954 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:19,954 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:19,954 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:19,955 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,955 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,956 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:19,956 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:19,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:19,956 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:19,957 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:19,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,957 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:19,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:19,958 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:19,958 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:19,958 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:20,376 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103361f556485c44684bd38f2016a2a1191 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103361f556485c44684bd38f2016a2a1191 2023-03-14 21:54:20,382 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/4fb62976233c4c3ea912b3a0c7261587 2023-03-14 21:54:20,387 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/4fb62976233c4c3ea912b3a0c7261587 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587 2023-03-14 21:54:20,392 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587, entries=1, sequenceid=33, filesize=4.9 K 2023-03-14 21:54:20,393 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 435ms, sequenceid=33, compaction requested=true 2023-03-14 21:54:20,393 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:20,393 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:20,395 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@106372e9[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:20,397 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:20,397 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,398 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:20,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:20,398 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:20,398 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,398 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:20,401 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:20,403 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:20,403 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:20,403 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:20,404 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,404 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:20,405 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,405 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,405 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,406 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:20,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:20,409 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:20,409 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:20,410 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:20,410 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:20,410 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:20,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:20,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:20,510 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:20,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:20,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:20,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:20,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,518 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:20,518 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:20,518 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:20,518 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:20,519 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,519 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,521 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:20,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:20,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:20,521 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,521 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,521 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:20,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:20,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,522 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,522 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:20,522 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,522 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:20,522 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:20,522 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:20,523 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:20,523 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:20,523 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:20,523 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:20,524 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:20,939 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907001a69d093994a20908a62038cc6b6d2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907001a69d093994a20908a62038cc6b6d2 2023-03-14 21:54:20,944 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/b8cf020e441b4458b2a09dcdf5853110 2023-03-14 21:54:20,950 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/b8cf020e441b4458b2a09dcdf5853110 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110 2023-03-14 21:54:20,955 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110, entries=1, sequenceid=37, filesize=4.9 K 2023-03-14 21:54:20,956 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 433ms, sequenceid=37, compaction requested=true 2023-03-14 21:54:20,957 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:20,957 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:20,957 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:20,959 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2339037c[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:20,961 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:20,961 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,965 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:20,965 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,966 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:20,971 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:20,973 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:20,973 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:20,973 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:20,973 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,973 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:20,976 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,976 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,976 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:20,977 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,978 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:20,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,980 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,980 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,980 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:20,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:20,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:20,980 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:20,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:20,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:20,980 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:20,981 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:20,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:20,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:20,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:21,081 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:21,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:21,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:21,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:21,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,089 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:21,089 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:21,089 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:21,089 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:21,090 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,090 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,091 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:21,091 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,092 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,092 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:21,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,093 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:21,093 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:21,093 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:21,094 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:21,094 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. started... 2023-03-14 21:54:21,094 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d56bd8e4f6c9c696d51e6aa54aa60658 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:21,510 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511206fce02bcaa4448f19cb41bc0950477f7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511206fce02bcaa4448f19cb41bc0950477f7 2023-03-14 21:54:21,517 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/c44090fe6b5f48ccbc926ff81cb8fbcf 2023-03-14 21:54:21,522 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/c44090fe6b5f48ccbc926ff81cb8fbcf as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf 2023-03-14 21:54:21,527 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf, entries=1, sequenceid=41, filesize=4.9 K 2023-03-14 21:54:21,528 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d56bd8e4f6c9c696d51e6aa54aa60658 in 434ms, sequenceid=41, compaction requested=true 2023-03-14 21:54:21,529 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:21,529 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:21,529 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:21,529 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:21,529 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,529 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:21,529 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:21,531 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,531 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:21,531 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,531 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:21,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:21,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:21,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@666e6070[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:21,533 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:21,533 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,535 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,536 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:21,536 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:21,536 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:21,536 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,536 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:21,537 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:21,537 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,537 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:21,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:21,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:21,537 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:21,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:21,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:21,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:21,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:21,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:21,540 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:21,540 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:21,540 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:21,540 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,540 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:21,542 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:21,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:21,542 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:21,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,543 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,546 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:21,546 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:21,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:21,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-14 21:54:21,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:21,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:21,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:21,546 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:21,546 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:21,547 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:21,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:21,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,548 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,647 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:21,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:21,649 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-14 21:54:21,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:21,652 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:21,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:21,653 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831461653 2023-03-14 21:54:21,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:54:21,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-14 21:54:21,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:21,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x29cf525b to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:21,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@353003c6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:21,669 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,135 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103298efcf1f6034953b600e45f8e05e13e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103298efcf1f6034953b600e45f8e05e13e 2023-03-14 21:54:22,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:22,155 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:22,156 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59298, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:22,166 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:22,170 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58652, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:22,182 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 first=Optional[r3] last=Optional[r6] 2023-03-14 21:54:22,182 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2}] 2023-03-14 21:54:22,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 for inclusion in d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:22,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-14 21:54:22,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:22,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:22,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ef6h58jnbjh12lafnos98te13f9pr37qm0ipsagor3vo7u2q7oplntt529mskdgc/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 2023-03-14 21:54:22,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ef6h58jnbjh12lafnos98te13f9pr37qm0ipsagor3vo7u2q7oplntt529mskdgc/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_ 2023-03-14 21:54:22,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ef6h58jnbjh12lafnos98te13f9pr37qm0ipsagor3vo7u2q7oplntt529mskdgc/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 into d56bd8e4f6c9c696d51e6aa54aa60658/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_ - updating store file list. 2023-03-14 21:54:22,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_ into d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:22,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ef6h58jnbjh12lafnos98te13f9pr37qm0ipsagor3vo7u2q7oplntt529mskdgc/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 into d56bd8e4f6c9c696d51e6aa54aa60658/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_) 2023-03-14 21:54:22,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ef6h58jnbjh12lafnos98te13f9pr37qm0ipsagor3vo7u2q7oplntt529mskdgc/family1/c8ce2ae8362042c9a20ee5c38c3e0fa2 2023-03-14 21:54:22,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:54:22,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:54:22,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): d56bd8e4f6c9c696d51e6aa54aa60658/family1 is initiating minor compaction 2023-03-14 21:54:22,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@77ff1dfe; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:22,212 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d56bd8e4f6c9c696d51e6aa54aa60658/family1 in testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:22,212 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp, totalSize=50.0 K 2023-03-14 21:54:22,212 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 8428c9c7dc354f2f9b81c3d5b4a05789, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:54:22,212 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting f2682ff14c214fa69428db5040f9d33b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5be4a460287143abbf72217dbe34b4af, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting cc2a344acec14995a607e9c1caeff3ef, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 1ddc1624f0664fba8b19e2f3449ace8f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 425dd4647b124419ba325fcf5e6ff862, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 4fb62976233c4c3ea912b3a0c7261587, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting b8cf020e441b4458b2a09dcdf5853110, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c44090fe6b5f48ccbc926ff81cb8fbcf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:54:22,213 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d8332d42abb24c50974185cd90308652_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:54:22,214 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:22,215 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:22,217 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d61a75d504574a3fbd6072d32aabb536 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d61a75d504574a3fbd6072d32aabb536 2023-03-14 21:54:22,219 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123193e20a7b7c3c46bab4ed69c797218266 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123193e20a7b7c3c46bab4ed69c797218266 2023-03-14 21:54:22,221 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010132c479970af1440f8ebbfffd5e0713f5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010132c479970af1440f8ebbfffd5e0713f5 2023-03-14 21:54:22,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103361f556485c44684bd38f2016a2a1191 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103361f556485c44684bd38f2016a2a1191 2023-03-14 21:54:22,222 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-14 21:54:22,222 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,223 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,270 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): d56bd8e4f6c9c696d51e6aa54aa60658#family1#compaction#34 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:22,682 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053959320b227448fc8449d7d6d6717adf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512053959320b227448fc8449d7d6d6717adf 2023-03-14 21:54:22,684 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:22,704 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/5918ac255f3146ab97cf794300f9c938 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5918ac255f3146ab97cf794300f9c938 2023-03-14 21:54:22,711 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d56bd8e4f6c9c696d51e6aa54aa60658/family1 of d56bd8e4f6c9c696d51e6aa54aa60658 into 5918ac255f3146ab97cf794300f9c938(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:54:22,711 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:22,712 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678830862212; duration=0sec 2023-03-14 21:54:22,712 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:22,721 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 first=Optional[r1] last=Optional[r2] 2023-03-14 21:54:22,723 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004}] 2023-03-14 21:54:22,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_] to archive 2023-03-14 21:54:22,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 for inclusion in d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:22,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:22,737 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/8428c9c7dc354f2f9b81c3d5b4a05789 2023-03-14 21:54:22,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-14 21:54:22,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:22,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:22,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/f2682ff14c214fa69428db5040f9d33b 2023-03-14 21:54:22,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__4gsq8u9387cedit98a405thhg50u577htqiqif0flama1b83v21dnqp4qhcnreg8/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 2023-03-14 21:54:22,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5be4a460287143abbf72217dbe34b4af 2023-03-14 21:54:22,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__4gsq8u9387cedit98a405thhg50u577htqiqif0flama1b83v21dnqp4qhcnreg8/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_ 2023-03-14 21:54:22,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/cc2a344acec14995a607e9c1caeff3ef 2023-03-14 21:54:22,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__4gsq8u9387cedit98a405thhg50u577htqiqif0flama1b83v21dnqp4qhcnreg8/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 into d56bd8e4f6c9c696d51e6aa54aa60658/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_ - updating store file list. 2023-03-14 21:54:22,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/1ddc1624f0664fba8b19e2f3449ace8f 2023-03-14 21:54:22,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/425dd4647b124419ba325fcf5e6ff862 2023-03-14 21:54:22,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/4fb62976233c4c3ea912b3a0c7261587 2023-03-14 21:54:22,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_ into d56bd8e4f6c9c696d51e6aa54aa60658/family1 2023-03-14 21:54:22,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__4gsq8u9387cedit98a405thhg50u577htqiqif0flama1b83v21dnqp4qhcnreg8/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 into d56bd8e4f6c9c696d51e6aa54aa60658/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_) 2023-03-14 21:54:22,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/b8cf020e441b4458b2a09dcdf5853110 2023-03-14 21:54:22,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithWeeklyPolicy__4gsq8u9387cedit98a405thhg50u577htqiqif0flama1b83v21dnqp4qhcnreg8/family1/0c2c9bd046a54eb2ac2cfbcc9c9be004 2023-03-14 21:54:22,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:54:22,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c44090fe6b5f48ccbc926ff81cb8fbcf 2023-03-14 21:54:22,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:54:22,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): d56bd8e4f6c9c696d51e6aa54aa60658/family1 is initiating minor compaction (all files) 2023-03-14 21:54:22,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64de8810; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:22,750 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d56bd8e4f6c9c696d51e6aa54aa60658/family1 in testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:54:22,750 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5918ac255f3146ab97cf794300f9c938, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp, totalSize=176.8 K 2023-03-14 21:54:22,752 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d2b4c176c9124587915617ab8143c7d2, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:54:22,759 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5918ac255f3146ab97cf794300f9c938, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:54:22,759 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:22,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d8332d42abb24c50974185cd90308652_SeqId_44_ 2023-03-14 21:54:22,759 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c8d878130fc6495bb83f81f595883d6e_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245591 2023-03-14 21:54:22,760 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:22,762 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201635b4621ee2843629ca3c384e8269853 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201635b4621ee2843629ca3c384e8269853 2023-03-14 21:54:22,763 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057ac0f9cd825448a38c1772164c02a90b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057ac0f9cd825448a38c1772164c02a90b 2023-03-14 21:54:22,763 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-14 21:54:22,763 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-14 21:54:22,764 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:22,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:22,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:22,765 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:22,767 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.1150 sec 2023-03-14 21:54:22,779 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): d56bd8e4f6c9c696d51e6aa54aa60658#family1#compaction#35 average throughput is 17.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:22,865 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=258 (was 249) - Thread LEAK? -, OpenFileDescriptor=601 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=261 (was 280), ProcessCount=169 (was 169), AvailableMemoryMB=3331 (was 3769) 2023-03-14 21:54:22,878 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=258, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=261, ProcessCount=169, AvailableMemoryMB=3319 2023-03-14 21:54:22,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-14 21:54:22,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:22,884 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:54:22,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-14 21:54:22,885 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:54:22,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-14 21:54:22,887 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:22,887 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:22,887 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb empty. 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0 empty. 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf empty. 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:22,888 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:22,889 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-14 21:54:22,898 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:22,899 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => aec273c00f5dacbd3228bda8e96041d0, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:22,900 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8940ea8e9d4ee2ca9167816f0547cdcb, NAME => 'testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:22,900 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => b326da14072a6bfe886e8eabccb45fdf, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:22,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-14 21:54:23,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-14 21:54:23,314 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,314 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing aec273c00f5dacbd3228bda8e96041d0, disabling compactions & flushes 2023-03-14 21:54:23,315 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. after waiting 0 ms 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,315 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing b326da14072a6bfe886e8eabccb45fdf, disabling compactions & flushes 2023-03-14 21:54:23,315 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,315 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. after waiting 0 ms 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,316 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for b326da14072a6bfe886e8eabccb45fdf: 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 8940ea8e9d4ee2ca9167816f0547cdcb, disabling compactions & flushes 2023-03-14 21:54:23,316 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. after waiting 0 ms 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,316 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,316 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 8940ea8e9d4ee2ca9167816f0547cdcb: 2023-03-14 21:54:23,319 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:54:23,320 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830863320"}]},"ts":"1678830863320"} 2023-03-14 21:54:23,320 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678830863320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830863320"}]},"ts":"1678830863320"} 2023-03-14 21:54:23,321 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830863320"}]},"ts":"1678830863320"} 2023-03-14 21:54:23,324 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-14 21:54:23,325 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:54:23,325 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830863325"}]},"ts":"1678830863325"} 2023-03-14 21:54:23,326 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-14 21:54:23,330 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8940ea8e9d4ee2ca9167816f0547cdcb, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b326da14072a6bfe886e8eabccb45fdf, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=aec273c00f5dacbd3228bda8e96041d0, ASSIGN}] 2023-03-14 21:54:23,332 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=aec273c00f5dacbd3228bda8e96041d0, ASSIGN 2023-03-14 21:54:23,332 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b326da14072a6bfe886e8eabccb45fdf, ASSIGN 2023-03-14 21:54:23,332 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8940ea8e9d4ee2ca9167816f0547cdcb, ASSIGN 2023-03-14 21:54:23,333 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=aec273c00f5dacbd3228bda8e96041d0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:23,333 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b326da14072a6bfe886e8eabccb45fdf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:23,333 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8940ea8e9d4ee2ca9167816f0547cdcb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:23,485 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=8940ea8e9d4ee2ca9167816f0547cdcb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,485 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=aec273c00f5dacbd3228bda8e96041d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,485 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b326da14072a6bfe886e8eabccb45fdf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,485 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863484"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830863484"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830863484"}]},"ts":"1678830863484"} 2023-03-14 21:54:23,485 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863485"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830863485"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830863485"}]},"ts":"1678830863485"} 2023-03-14 21:54:23,485 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678830863485"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830863485"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830863485"}]},"ts":"1678830863485"} 2023-03-14 21:54:23,487 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure 8940ea8e9d4ee2ca9167816f0547cdcb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:23,488 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure aec273c00f5dacbd3228bda8e96041d0, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:23,488 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure b326da14072a6bfe886e8eabccb45fdf, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:23,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-14 21:54:23,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b326da14072a6bfe886e8eabccb45fdf, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.', STARTKEY => '1', ENDKEY => '2'} 2023-03-14 21:54:23,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,643 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,644 DEBUG [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/family1 2023-03-14 21:54:23,644 DEBUG [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/family1 2023-03-14 21:54:23,645 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b326da14072a6bfe886e8eabccb45fdf columnFamilyName family1 2023-03-14 21:54:23,646 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] regionserver.HStore(310): Store=b326da14072a6bfe886e8eabccb45fdf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,646 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,647 DEBUG [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/family2 2023-03-14 21:54:23,647 DEBUG [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/family2 2023-03-14 21:54:23,648 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b326da14072a6bfe886e8eabccb45fdf columnFamilyName family2 2023-03-14 21:54:23,649 INFO [StoreOpener-b326da14072a6bfe886e8eabccb45fdf-1] regionserver.HStore(310): Store=b326da14072a6bfe886e8eabccb45fdf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:54:23,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:54:23,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:23,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b326da14072a6bfe886e8eabccb45fdf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63434580, jitterRate=-0.05475109815597534}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:54:23,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b326da14072a6bfe886e8eabccb45fdf: 2023-03-14 21:54:23,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf., pid=33, masterSystemTime=1678830863638 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:23,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8940ea8e9d4ee2ca9167816f0547cdcb, NAME => 'testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.', STARTKEY => '', ENDKEY => '1'} 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,658 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b326da14072a6bfe886e8eabccb45fdf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,659 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678830863658"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830863658"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830863658"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830863658"}]},"ts":"1678830863658"} 2023-03-14 21:54:23,659 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,661 DEBUG [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/family1 2023-03-14 21:54:23,661 DEBUG [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/family1 2023-03-14 21:54:23,661 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8940ea8e9d4ee2ca9167816f0547cdcb columnFamilyName family1 2023-03-14 21:54:23,662 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] regionserver.HStore(310): Store=8940ea8e9d4ee2ca9167816f0547cdcb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,662 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-14 21:54:23,662 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,662 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure b326da14072a6bfe886e8eabccb45fdf, server=jenkins-hbase4.apache.org,35143,1678830821197 in 172 msec 2023-03-14 21:54:23,664 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b326da14072a6bfe886e8eabccb45fdf, ASSIGN in 332 msec 2023-03-14 21:54:23,664 DEBUG [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/family2 2023-03-14 21:54:23,664 DEBUG [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/family2 2023-03-14 21:54:23,665 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8940ea8e9d4ee2ca9167816f0547cdcb columnFamilyName family2 2023-03-14 21:54:23,665 INFO [StoreOpener-8940ea8e9d4ee2ca9167816f0547cdcb-1] regionserver.HStore(310): Store=8940ea8e9d4ee2ca9167816f0547cdcb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:54:23,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:54:23,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:23,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8940ea8e9d4ee2ca9167816f0547cdcb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63977698, jitterRate=-0.04665800929069519}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:54:23,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8940ea8e9d4ee2ca9167816f0547cdcb: 2023-03-14 21:54:23,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb., pid=31, masterSystemTime=1678830863638 2023-03-14 21:54:23,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:23,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aec273c00f5dacbd3228bda8e96041d0, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.', STARTKEY => '2', ENDKEY => ''} 2023-03-14 21:54:23,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,674 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=8940ea8e9d4ee2ca9167816f0547cdcb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:23,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,675 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863674"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830863674"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830863674"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830863674"}]},"ts":"1678830863674"} 2023-03-14 21:54:23,676 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,678 DEBUG [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1 2023-03-14 21:54:23,678 DEBUG [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1 2023-03-14 21:54:23,678 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aec273c00f5dacbd3228bda8e96041d0 columnFamilyName family1 2023-03-14 21:54:23,678 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-14 21:54:23,678 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure 8940ea8e9d4ee2ca9167816f0547cdcb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 189 msec 2023-03-14 21:54:23,679 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] regionserver.HStore(310): Store=aec273c00f5dacbd3228bda8e96041d0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,679 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8940ea8e9d4ee2ca9167816f0547cdcb, ASSIGN in 349 msec 2023-03-14 21:54:23,681 DEBUG [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family2 2023-03-14 21:54:23,681 DEBUG [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family2 2023-03-14 21:54:23,681 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aec273c00f5dacbd3228bda8e96041d0 columnFamilyName family2 2023-03-14 21:54:23,682 INFO [StoreOpener-aec273c00f5dacbd3228bda8e96041d0-1] regionserver.HStore(310): Store=aec273c00f5dacbd3228bda8e96041d0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:23,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:54:23,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:54:23,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:23,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aec273c00f5dacbd3228bda8e96041d0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73413867, jitterRate=0.09395186603069305}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:54:23,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:23,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0., pid=32, masterSystemTime=1678830863638 2023-03-14 21:54:23,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:23,692 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=aec273c00f5dacbd3228bda8e96041d0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:23,692 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678830863692"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830863692"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830863692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830863692"}]},"ts":"1678830863692"} 2023-03-14 21:54:23,696 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-14 21:54:23,696 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure aec273c00f5dacbd3228bda8e96041d0, server=jenkins-hbase4.apache.org,35143,1678830821197 in 207 msec 2023-03-14 21:54:23,697 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-14 21:54:23,697 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=aec273c00f5dacbd3228bda8e96041d0, ASSIGN in 366 msec 2023-03-14 21:54:23,698 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:54:23,698 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830863698"}]},"ts":"1678830863698"} 2023-03-14 21:54:23,699 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-14 21:54:23,702 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:54:23,703 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 821 msec 2023-03-14 21:54:23,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-14 21:54:23,990 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-14 21:54:23,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:23,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,000 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-14 21:54:24,000 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:24,000 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-14 21:54:24,000 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-14 21:54:24,000 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,000 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,003 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:24,003 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:24,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:24,003 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,003 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,004 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,004 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/.tmp/family1/52965dc36e144a12a2bc251fbf42884c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/52965dc36e144a12a2bc251fbf42884c 2023-03-14 21:54:24,005 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-14 21:54:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-14 21:54:24,005 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:24,005 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. started... 2023-03-14 21:54:24,006 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aec273c00f5dacbd3228bda8e96041d0 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-14 21:54:24,006 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:54:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:24,006 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. started... 2023-03-14 21:54:24,006 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:24,007 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8940ea8e9d4ee2ca9167816f0547cdcb: 2023-03-14 21:54:24,007 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. started... 2023-03-14 21:54:24,007 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:24,008 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b326da14072a6bfe886e8eabccb45fdf: 2023-03-14 21:54:24,008 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:54:24,008 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:24,008 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:54:24,013 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d56bd8e4f6c9c696d51e6aa54aa60658/family1 of d56bd8e4f6c9c696d51e6aa54aa60658 into 52965dc36e144a12a2bc251fbf42884c(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:54:24,013 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:54:24,013 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1678830862750; duration=1sec 2023-03-14 21:54:24,013 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:24,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5918ac255f3146ab97cf794300f9c938, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_] to archive 2023-03-14 21:54:24,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:24,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/d2b4c176c9124587915617ab8143c7d2 2023-03-14 21:54:24,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5918ac255f3146ab97cf794300f9c938 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/5918ac255f3146ab97cf794300f9c938 2023-03-14 21:54:24,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/family1/c8d878130fc6495bb83f81f595883d6e_SeqId_47_ 2023-03-14 21:54:24,429 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303149d0a940e18ec48dfbc2ce25ff30b0949 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303149d0a940e18ec48dfbc2ce25ff30b0949 2023-03-14 21:54:24,435 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp/family1/5de98fe85ec34e7d98b79564ce8dab15 2023-03-14 21:54:24,440 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp/family1/5de98fe85ec34e7d98b79564ce8dab15 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15 2023-03-14 21:54:24,445 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15, entries=1, sequenceid=5, filesize=4.9 K 2023-03-14 21:54:24,446 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for aec273c00f5dacbd3228bda8e96041d0 in 440ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:24,446 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-14 21:54:24,446 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:24,446 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:24,446 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:54:24,447 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:54:24,447 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,447 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-14 21:54:24,447 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-14 21:54:24,448 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,448 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:24,449 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:24,449 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:24,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-14 21:54:24,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62df744c[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:24,451 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-14 21:54:24,451 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,452 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,452 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-14 21:54:24,452 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-14 21:54:24,452 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:24,452 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,452 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:24,453 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,453 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-14 21:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:24,454 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:24,454 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-14 21:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-14 21:54:24,456 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-14 21:54:24,456 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:24,456 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:24,456 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,456 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:24,459 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,459 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,459 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:24,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:24,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:24,460 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:24,461 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:24,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:24,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-14 21:54:24,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:24,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:24,466 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:24,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-14 21:54:24,467 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:24,467 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:24,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:24,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:24,567 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:24,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:24,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:24,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:24,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,574 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-14 21:54:24,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:24,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-14 21:54:24,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-14 21:54:24,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,576 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:24,576 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:24,576 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:24,576 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-14 21:54:24,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:24,577 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-14 21:54:24,577 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:24,577 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. started... 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:24,578 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aec273c00f5dacbd3228bda8e96041d0 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-14 21:54:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. started... 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. started... 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8940ea8e9d4ee2ca9167816f0547cdcb: 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b326da14072a6bfe886e8eabccb45fdf: 2023-03-14 21:54:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:54:24,578 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:54:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:54:24,991 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314d8a16abf8b0f4162beab28a8c9583ad7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230314d8a16abf8b0f4162beab28a8c9583ad7 2023-03-14 21:54:25,397 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp/family1/56b22dc6533446578bd98a212ef3970c 2023-03-14 21:54:25,403 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp/family1/56b22dc6533446578bd98a212ef3970c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c 2023-03-14 21:54:25,407 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c, entries=2, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:25,408 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for aec273c00f5dacbd3228bda8e96041d0 in 830ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:25,408 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:25,408 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:25,409 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:54:25,409 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:54:25,409 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,409 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-14 21:54:25,409 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-14 21:54:25,411 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,411 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:25,412 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,412 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:25,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-14 21:54:25,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69a0a6c4[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:25,414 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-14 21:54:25,414 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,415 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,415 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-14 21:54:25,415 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-14 21:54:25,415 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:25,415 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,415 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:25,416 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,416 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-14 21:54:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:25,416 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:25,417 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-14 21:54:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:25,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:25,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-14 21:54:25,418 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-14 21:54:25,419 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:25,419 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:25,419 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,419 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:25,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:25,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,420 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:25,421 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,424 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:25,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,427 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:25,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-14 21:54:25,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:25,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:25,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-14 21:54:25,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:25,427 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:25,528 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:25,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:25,535 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-14 21:54:25,537 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-14 21:54:25,538 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-14 21:54:25,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:25,539 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831465539 2023-03-14 21:54:25,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:25,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-14 21:54:25,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-14 21:54:25,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x4d2c9136 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:25,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78b79566, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:25,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230314 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-14 21:54:26,375 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031483d70df02a734e66bdc304ac2e468be8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023031483d70df02a734e66bdc304ac2e468be8 2023-03-14 21:54:26,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-14 21:54:26,389 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:26,390 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42396, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:26,397 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:26,399 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48596, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:26,411 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230314/family1/5b3066aeb4d244a0a5c9b9f1c63c263b first=Optional[k0] last=Optional[k1] 2023-03-14 21:54:26,412 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230314/family1/5b3066aeb4d244a0a5c9b9f1c63c263b}] 2023-03-14 21:54:26,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230314/family1/5b3066aeb4d244a0a5c9b9f1c63c263b for inclusion in aec273c00f5dacbd3228bda8e96041d0/family1 2023-03-14 21:54:26,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-14 21:54:26,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-14 21:54:26,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:26,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230314/family1/5b3066aeb4d244a0a5c9b9f1c63c263b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testScannerOnBulkLoadRefHFiles__cshfhcpu89n6klaq46gssbp3nirkm65fq24u9ebhkplrqbk0d2gdr5mhnf026k54/family1/5b3066aeb4d244a0a5c9b9f1c63c263b 2023-03-14 21:54:26,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testScannerOnBulkLoadRefHFiles__cshfhcpu89n6klaq46gssbp3nirkm65fq24u9ebhkplrqbk0d2gdr5mhnf026k54/family1/5b3066aeb4d244a0a5c9b9f1c63c263b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_ 2023-03-14 21:54:26,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testScannerOnBulkLoadRefHFiles__cshfhcpu89n6klaq46gssbp3nirkm65fq24u9ebhkplrqbk0d2gdr5mhnf026k54/family1/5b3066aeb4d244a0a5c9b9f1c63c263b into aec273c00f5dacbd3228bda8e96041d0/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_ - updating store file list. 2023-03-14 21:54:26,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_ into aec273c00f5dacbd3228bda8e96041d0/family1 2023-03-14 21:54:26,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testScannerOnBulkLoadRefHFiles__cshfhcpu89n6klaq46gssbp3nirkm65fq24u9ebhkplrqbk0d2gdr5mhnf026k54/family1/5b3066aeb4d244a0a5c9b9f1c63c263b into aec273c00f5dacbd3228bda8e96041d0/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_) 2023-03-14 21:54:26,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testScannerOnBulkLoadRefHFiles__cshfhcpu89n6klaq46gssbp3nirkm65fq24u9ebhkplrqbk0d2gdr5mhnf026k54/family1/5b3066aeb4d244a0a5c9b9f1c63c263b 2023-03-14 21:54:26,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:54:26,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:54:26,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): aec273c00f5dacbd3228bda8e96041d0/family1 is initiating minor compaction (all files) 2023-03-14 21:54:26,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c2dc1ab; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:26,430 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aec273c00f5dacbd3228bda8e96041d0/family1 in testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:54:26,430 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp, totalSize=15.1 K 2023-03-14 21:54:26,431 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5de98fe85ec34e7d98b79564ce8dab15, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830863991 2023-03-14 21:54:26,431 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 56b22dc6533446578bd98a212ef3970c, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830863991 2023-03-14 21:54:26,432 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 44e85f0b5ba242d6935be1d640570d5b_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830863991 2023-03-14 21:54:26,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-14 21:54:26,433 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:26,434 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303149d0a940e18ec48dfbc2ce25ff30b0949 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303149d0a940e18ec48dfbc2ce25ff30b0949 2023-03-14 21:54:26,435 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230314d8a16abf8b0f4162beab28a8c9583ad7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230314d8a16abf8b0f4162beab28a8c9583ad7 2023-03-14 21:54:26,435 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-14 21:54:26,435 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230314' 2023-03-14 21:54:26,436 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230314 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-14 21:54:26,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-14 21:54:26,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:26,437 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-14 21:54:26,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 901 msec 2023-03-14 21:54:26,443 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): aec273c00f5dacbd3228bda8e96041d0#family1#compaction#38 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:26,471 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=260 (was 258) - Thread LEAK? -, OpenFileDescriptor=607 (was 601) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=256 (was 261), ProcessCount=169 (was 169), AvailableMemoryMB=3261 (was 3319) 2023-03-14 21:54:26,482 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=260, OpenFileDescriptor=607, MaxFileDescriptor=60000, SystemLoadAverage=256, ProcessCount=169, AvailableMemoryMB=3260 2023-03-14 21:54:26,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:54:26,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:26,487 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:54:26,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-14 21:54:26,487 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:54:26,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-14 21:54:26,489 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:26,489 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3 empty. 2023-03-14 21:54:26,490 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:26,490 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-14 21:54:26,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-14 21:54:26,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-14 21:54:26,899 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:26,900 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => e05811c07f91e926119a25e5280a90f3, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:26,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:26,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing e05811c07f91e926119a25e5280a90f3, disabling compactions & flushes 2023-03-14 21:54:26,906 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:26,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:26,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. after waiting 0 ms 2023-03-14 21:54:26,907 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:26,907 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:26,907 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:26,909 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:54:26,910 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830866910"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830866910"}]},"ts":"1678830866910"} 2023-03-14 21:54:26,911 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:54:26,912 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:54:26,912 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830866912"}]},"ts":"1678830866912"} 2023-03-14 21:54:26,913 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-14 21:54:26,916 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e05811c07f91e926119a25e5280a90f3, ASSIGN}] 2023-03-14 21:54:26,918 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e05811c07f91e926119a25e5280a90f3, ASSIGN 2023-03-14 21:54:26,919 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e05811c07f91e926119a25e5280a90f3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:27,070 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e05811c07f91e926119a25e5280a90f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:27,070 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830867070"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830867070"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830867070"}]},"ts":"1678830867070"} 2023-03-14 21:54:27,072 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure e05811c07f91e926119a25e5280a90f3, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:27,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-14 21:54:27,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:27,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e05811c07f91e926119a25e5280a90f3, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:27,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:27,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,229 INFO [StoreOpener-e05811c07f91e926119a25e5280a90f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,230 DEBUG [StoreOpener-e05811c07f91e926119a25e5280a90f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:27,230 DEBUG [StoreOpener-e05811c07f91e926119a25e5280a90f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:27,231 INFO [StoreOpener-e05811c07f91e926119a25e5280a90f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e05811c07f91e926119a25e5280a90f3 columnFamilyName family1 2023-03-14 21:54:27,232 INFO [StoreOpener-e05811c07f91e926119a25e5280a90f3-1] regionserver.HStore(310): Store=e05811c07f91e926119a25e5280a90f3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:27,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e05811c07f91e926119a25e5280a90f3 2023-03-14 21:54:27,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:27,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e05811c07f91e926119a25e5280a90f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71591535, jitterRate=0.06679700314998627}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:27,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:27,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3., pid=37, masterSystemTime=1678830867224 2023-03-14 21:54:27,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:27,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:27,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=e05811c07f91e926119a25e5280a90f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:27,243 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678830867242"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830867242"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830867242"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830867242"}]},"ts":"1678830867242"} 2023-03-14 21:54:27,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-14 21:54:27,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure e05811c07f91e926119a25e5280a90f3, server=jenkins-hbase4.apache.org,35143,1678830821197 in 172 msec 2023-03-14 21:54:27,247 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-14 21:54:27,247 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=e05811c07f91e926119a25e5280a90f3, ASSIGN in 329 msec 2023-03-14 21:54:27,248 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:54:27,248 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830867248"}]},"ts":"1678830867248"} 2023-03-14 21:54:27,249 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-14 21:54:27,252 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:54:27,254 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 768 msec 2023-03-14 21:54:27,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-14 21:54:27,592 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-14 21:54:27,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:27,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,623 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:27,623 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:27,623 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:27,623 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:27,624 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,624 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,626 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:27,626 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:27,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:27,626 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:27,626 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:27,627 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:27,628 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:27,628 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:27,628 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:27,628 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:27,629 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:54:27,664 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/.tmp/family1/2953c14073dc4916a1ecae0e6d123eb9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/2953c14073dc4916a1ecae0e6d123eb9 2023-03-14 21:54:27,668 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aec273c00f5dacbd3228bda8e96041d0/family1 of aec273c00f5dacbd3228bda8e96041d0 into 2953c14073dc4916a1ecae0e6d123eb9(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:54:27,669 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:54:27,669 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678830866430; duration=1sec 2023-03-14 21:54:27,669 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:27,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_] to archive 2023-03-14 21:54:27,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:27,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/5de98fe85ec34e7d98b79564ce8dab15 2023-03-14 21:54:27,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/56b22dc6533446578bd98a212ef3970c 2023-03-14 21:54:27,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/family1/44e85f0b5ba242d6935be1d640570d5b_SeqId_12_ 2023-03-14 21:54:28,051 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113003609bbc1941431a84a72a60046612f9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113003609bbc1941431a84a72a60046612f9 2023-03-14 21:54:28,455 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/e6095325df4049bea1b47d508cb3d7e3 2023-03-14 21:54:28,460 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/e6095325df4049bea1b47d508cb3d7e3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3 2023-03-14 21:54:28,464 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-14 21:54:28,465 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 837ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:28,465 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:28,466 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:28,466 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:28,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:28,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:28,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:28,466 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:28,468 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,468 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:28,468 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,468 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2696a6f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:28,469 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:28,470 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,470 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,471 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:28,471 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:28,471 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:28,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:28,472 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:28,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:28,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:28,474 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:28,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:28,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:28,474 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,474 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:28,476 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,476 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,476 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:28,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:28,477 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,478 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:28,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:28,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:28,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:28,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:28,480 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:28,480 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,481 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,481 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,481 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:28,481 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:28,581 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:28,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:28,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:28,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:28,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,588 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:28,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:28,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:28,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:28,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,588 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,589 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:28,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:28,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:28,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:28,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:28,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:28,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:28,590 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:28,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:28,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:28,591 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:28,591 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:28,591 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:54:29,003 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b008878e647744079812c21cf4c3647d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128b008878e647744079812c21cf4c3647d 2023-03-14 21:54:29,172 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:54:29,410 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/7706a3be75434b6d8135d8f95552cb4f 2023-03-14 21:54:29,415 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/7706a3be75434b6d8135d8f95552cb4f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f 2023-03-14 21:54:29,419 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:29,420 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 829ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:29,420 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:29,420 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:29,420 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:29,420 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:29,420 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:29,421 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:29,422 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,422 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:29,422 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,422 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:29,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:29,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1711f7b2[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:29,424 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:29,424 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,431 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:29,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:29,431 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,431 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:29,431 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:29,435 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:29,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:29,437 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:29,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:29,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:29,438 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,438 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:29,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:29,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:29,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:29,439 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:29,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,441 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:29,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,444 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:29,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:29,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:29,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:29,445 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:29,445 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:29,444 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:29,545 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:29,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:29,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:29,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:29,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,552 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:29,552 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:29,552 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:29,552 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:29,552 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,552 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,553 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:29,554 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:29,554 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:29,554 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,555 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:29,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:29,555 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:29,555 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:29,555 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:29,969 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120163a54e67fe8b4932a6a089a16fc7514d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120163a54e67fe8b4932a6a089a16fc7514d 2023-03-14 21:54:30,375 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/f5bc52ea22ed4da5803df141cb4ca036 2023-03-14 21:54:30,380 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/f5bc52ea22ed4da5803df141cb4ca036 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036 2023-03-14 21:54:30,384 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036, entries=1, sequenceid=13, filesize=4.9 K 2023-03-14 21:54:30,385 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 830ms, sequenceid=13, compaction requested=true 2023-03-14 21:54:30,385 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:30,385 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:30,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:30,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:30,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:30,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:30,392 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,392 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:30,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,393 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:30,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:30,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:30,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e758c08[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:30,394 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:30,394 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,396 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,397 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:30,397 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:30,397 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:30,397 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,397 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:30,399 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:30,399 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,399 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:30,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:30,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:30,399 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:30,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:30,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:30,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:30,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:30,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:30,401 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:30,402 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:30,402 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:30,402 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,402 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:30,403 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,403 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:30,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:30,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,403 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:30,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:30,404 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:30,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,405 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:30,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:30,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:30,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:30,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:30,407 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:30,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:30,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,407 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,408 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:30,408 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:30,508 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:30,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:30,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:30,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:30,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,514 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:30,514 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:30,514 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:30,514 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:30,515 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,515 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,516 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,516 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:30,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:30,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:30,516 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:30,516 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:30,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:30,517 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:30,517 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:30,518 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:30,518 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:30,518 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:30,931 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e61c6311926642f3bcf95c2c251150b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61c6311926642f3bcf95c2c251150b0 2023-03-14 21:54:31,336 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/fb87ae45f5aa4a6681997c26a6c1300b 2023-03-14 21:54:31,342 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/fb87ae45f5aa4a6681997c26a6c1300b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b 2023-03-14 21:54:31,346 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b, entries=1, sequenceid=17, filesize=4.9 K 2023-03-14 21:54:31,347 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 829ms, sequenceid=17, compaction requested=true 2023-03-14 21:54:31,348 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:31,348 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:31,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:31,350 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,350 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@628f8d84[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:31,351 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:31,351 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,352 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,352 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:31,352 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:31,352 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:31,353 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,353 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:31,354 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:31,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:31,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:31,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:31,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:31,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:31,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:31,359 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,359 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,359 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:31,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:31,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:31,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:31,360 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,361 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:31,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:31,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,363 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:31,363 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:31,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:31,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:31,463 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:31,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:31,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:31,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:31,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,470 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:31,470 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:31,470 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:31,470 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:31,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,472 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,472 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:31,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:31,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:31,473 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:31,473 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,473 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:31,474 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:31,474 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:31,474 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:31,887 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286c2d0d8285304f718236e10671188a46 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512286c2d0d8285304f718236e10671188a46 2023-03-14 21:54:32,293 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/413e0f9996e54996b69ed0af14b81f54 2023-03-14 21:54:32,298 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/413e0f9996e54996b69ed0af14b81f54 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54 2023-03-14 21:54:32,302 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54, entries=1, sequenceid=21, filesize=4.9 K 2023-03-14 21:54:32,303 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 829ms, sequenceid=21, compaction requested=true 2023-03-14 21:54:32,303 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:32,303 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:32,303 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:32,303 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:32,304 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,304 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:32,304 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:32,308 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,308 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:32,309 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,309 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:32,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:32,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b7cb9a8[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:32,310 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:32,310 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,311 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,311 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:32,311 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:32,311 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,311 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:32,311 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:32,313 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:32,313 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,313 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,313 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:32,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:32,315 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:32,315 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:32,315 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:32,315 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,315 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:32,316 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,316 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,316 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:32,317 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,318 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:32,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,321 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:32,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:32,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:32,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:32,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:32,322 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:32,322 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:32,321 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:32,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:32,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:32,422 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:32,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:32,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:32,449 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:32,449 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d2c9136 to 127.0.0.1:58882 2023-03-14 21:54:32,449 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:32,449 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:32,449 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29cf525b to 127.0.0.1:58882 2023-03-14 21:54:32,449 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:32,450 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:32,450 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2f08c329 to 127.0.0.1:58882 2023-03-14 21:54:32,450 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:32,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:32,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,450 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:32,450 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:32,451 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:32,451 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:32,452 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,452 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,453 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:32,453 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:32,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:32,454 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:32,454 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:32,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:32,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:32,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:32,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:32,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:32,455 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:32,455 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:32,455 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:32,868 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231aebe10d1f26a40cd86de66eb4be424d1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231aebe10d1f26a40cd86de66eb4be424d1 2023-03-14 21:54:33,273 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/eb7fce9c4fd64c49bf6fb437d454eb60 2023-03-14 21:54:33,278 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/eb7fce9c4fd64c49bf6fb437d454eb60 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60 2023-03-14 21:54:33,282 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60, entries=1, sequenceid=25, filesize=4.9 K 2023-03-14 21:54:33,283 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 828ms, sequenceid=25, compaction requested=true 2023-03-14 21:54:33,283 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:33,283 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:33,283 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:33,283 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:33,283 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,284 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:33,284 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:33,286 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,286 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,286 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,286 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:33,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:33,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33c87712[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:33,287 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:33,288 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,289 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,289 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:33,289 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:33,289 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:33,289 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,289 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:33,290 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:33,290 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,290 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:33,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,290 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:33,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:33,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:33,293 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:33,293 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:33,293 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:33,293 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,293 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:33,294 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,294 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,294 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:33,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:33,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:33,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,300 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:33,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:33,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:33,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:33,303 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:33,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:33,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:33,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:33,404 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:33,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:33,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:33,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:33,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,411 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:33,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:33,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:33,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:33,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,413 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:33,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:33,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:33,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:33,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:33,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:33,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:33,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:33,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:33,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:33,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:33,414 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:33,414 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:33,414 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:33,426 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014fb906cfbce54ffa8140bd8c444e0a65 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fb906cfbce54ffa8140bd8c444e0a65 2023-03-14 21:54:33,833 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/3366fb78748e4a99a4b2401c2db8536f 2023-03-14 21:54:33,838 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/3366fb78748e4a99a4b2401c2db8536f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f 2023-03-14 21:54:33,842 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f, entries=1, sequenceid=29, filesize=4.9 K 2023-03-14 21:54:33,843 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 429ms, sequenceid=29, compaction requested=true 2023-03-14 21:54:33,843 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:33,843 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:33,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:33,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:33,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:33,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:33,845 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,845 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,845 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,845 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:33,847 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:33,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11ffa8af[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:33,847 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,848 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,848 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:33,848 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:33,848 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:33,848 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,848 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:33,851 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:33,851 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,851 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,851 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:33,853 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:33,853 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:33,853 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:33,853 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:33,853 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,853 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:33,854 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,854 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:33,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:33,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,854 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:33,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:33,855 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:33,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,856 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:33,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,863 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,863 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,863 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:33,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:33,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:33,864 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:33,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:33,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:33,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:33,865 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:33,865 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:33,965 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:33,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:33,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:33,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:33,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,972 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:33,972 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:33,972 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:33,972 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:33,972 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,972 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,973 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,973 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:33,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:33,974 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:33,974 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,975 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:33,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:33,975 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:33,975 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:33,975 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:33,975 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:33,975 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:33,975 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:33,975 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:33,975 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:34,388 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010381dfb755e9f845a290a6eae0d0827af1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010381dfb755e9f845a290a6eae0d0827af1 2023-03-14 21:54:34,794 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/2d2982f605474727a4e8a3519924858c 2023-03-14 21:54:34,798 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/2d2982f605474727a4e8a3519924858c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c 2023-03-14 21:54:34,802 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c, entries=1, sequenceid=33, filesize=4.9 K 2023-03-14 21:54:34,803 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 828ms, sequenceid=33, compaction requested=true 2023-03-14 21:54:34,803 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:34,803 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:34,803 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:34,803 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:34,803 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,803 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:34,803 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:34,805 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,805 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:34,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:34,805 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:34,805 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:34,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:34,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:34,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:34,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f82e4b9[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:34,807 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:34,807 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,809 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,809 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:34,809 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:34,809 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:34,809 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,810 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:34,811 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:34,811 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:34,811 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,811 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:34,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:34,813 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:34,813 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:34,813 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:34,813 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,813 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:34,815 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,815 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:34,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,815 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:34,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:34,815 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:34,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:34,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:34,816 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:34,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:34,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:34,818 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,818 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:34,819 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,819 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:34,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:34,819 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:34,824 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:34,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:34,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:34,919 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:34,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:34,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:34,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:34,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,926 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:34,926 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:34,926 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:34,926 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:34,927 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,927 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,929 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:34,929 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:34,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:34,929 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:34,929 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:34,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:34,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:34,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:34,930 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:34,930 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:34,930 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:34,931 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:34,931 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:34,931 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:34,931 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:34,931 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:35,344 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090785caa7392b664acdb0e69ffb6e44936f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090785caa7392b664acdb0e69ffb6e44936f 2023-03-14 21:54:35,750 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/9c194c04d6bc4d6a845b97f2ff7bf402 2023-03-14 21:54:35,755 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/9c194c04d6bc4d6a845b97f2ff7bf402 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402 2023-03-14 21:54:35,759 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402, entries=1, sequenceid=37, filesize=4.9 K 2023-03-14 21:54:35,760 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 829ms, sequenceid=37, compaction requested=true 2023-03-14 21:54:35,760 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:35,760 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:35,762 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,762 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:35,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:35,762 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,762 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:35,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:35,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:35,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:35,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:35,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41fdbd18[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:35,763 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:35,763 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,764 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,765 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,765 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:35,766 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,766 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:35,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:35,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:35,766 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:35,766 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:35,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:35,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:35,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:35,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:35,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:35,768 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:35,768 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:35,768 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:35,768 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,768 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:35,770 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,770 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,770 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:35,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:35,771 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:35,772 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:35,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:35,774 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:35,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:35,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:35,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,874 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:35,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:35,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:35,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:35,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,881 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:35,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:35,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-14 21:54:35,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-14 21:54:35,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,882 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,883 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:35,883 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:35,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:35,883 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:35,883 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:35,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:35,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,884 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,884 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-14 21:54:35,884 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:35,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-14 21:54:35,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:35,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-14 21:54:35,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:35,884 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:35,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:35,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. started... 2023-03-14 21:54:35,884 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e05811c07f91e926119a25e5280a90f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:36,297 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112092c92c5705c54073a0f8442febf2dd76 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112092c92c5705c54073a0f8442febf2dd76 2023-03-14 21:54:36,302 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/f23ab9f2005a4222b132fb93352dda7d 2023-03-14 21:54:36,307 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/f23ab9f2005a4222b132fb93352dda7d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d 2023-03-14 21:54:36,311 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d, entries=1, sequenceid=41, filesize=4.9 K 2023-03-14 21:54:36,312 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e05811c07f91e926119a25e5280a90f3 in 428ms, sequenceid=41, compaction requested=true 2023-03-14 21:54:36,312 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:36,312 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-14 21:54:36,312 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-14 21:54:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,314 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,314 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:36,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-14 21:54:36,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31fd22d2[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:36,315 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-14 21:54:36,316 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-14 21:54:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-14 21:54:36,318 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:36,318 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,318 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:36,319 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-14 21:54:36,319 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,319 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,319 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-14 21:54:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:36,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:36,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:54:36,321 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-14 21:54:36,321 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:36,321 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:36,321 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,321 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:36,323 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,323 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:36,323 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:36,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:36,323 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,324 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:36,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-14 21:54:36,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:36,330 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:36,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:36,330 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,331 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,331 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:36,331 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:36,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:36,331 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:36,431 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:36,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:36,433 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-14 21:54:36,435 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:36,436 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:36,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:36,437 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831476437 2023-03-14 21:54:36,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:36,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-14 21:54:36,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:36,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x5f59b769 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:36,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b810541, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:36,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,280 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fa326c7f4bf84f998289bb1d8baed753 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103fa326c7f4bf84f998289bb1d8baed753 2023-03-14 21:54:37,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:37,294 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:37,295 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49770, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:37,301 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:37,303 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48946, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:37,312 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d9b6fdb343c246e0b301b8b0c7b0b563 first=Optional[r3] last=Optional[r6] 2023-03-14 21:54:37,313 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d9b6fdb343c246e0b301b8b0c7b0b563}] 2023-03-14 21:54:37,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d9b6fdb343c246e0b301b8b0c7b0b563 for inclusion in e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:37,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-14 21:54:37,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:37,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:37,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d9b6fdb343c246e0b301b8b0c7b0b563 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__elqsertj52tqdi7v0dagarf77ikuoi0io7b59tqjgat5h3q3ulbcte51khg480hq/family1/d9b6fdb343c246e0b301b8b0c7b0b563 2023-03-14 21:54:37,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__elqsertj52tqdi7v0dagarf77ikuoi0io7b59tqjgat5h3q3ulbcte51khg480hq/family1/d9b6fdb343c246e0b301b8b0c7b0b563 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_ 2023-03-14 21:54:37,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__elqsertj52tqdi7v0dagarf77ikuoi0io7b59tqjgat5h3q3ulbcte51khg480hq/family1/d9b6fdb343c246e0b301b8b0c7b0b563 into e05811c07f91e926119a25e5280a90f3/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_ - updating store file list. 2023-03-14 21:54:37,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_ into e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:37,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__elqsertj52tqdi7v0dagarf77ikuoi0io7b59tqjgat5h3q3ulbcte51khg480hq/family1/d9b6fdb343c246e0b301b8b0c7b0b563 into e05811c07f91e926119a25e5280a90f3/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_) 2023-03-14 21:54:37,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__elqsertj52tqdi7v0dagarf77ikuoi0io7b59tqjgat5h3q3ulbcte51khg480hq/family1/d9b6fdb343c246e0b301b8b0c7b0b563 2023-03-14 21:54:37,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:54:37,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:54:37,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): e05811c07f91e926119a25e5280a90f3/family1 is initiating minor compaction 2023-03-14 21:54:37,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6445056c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:37,329 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e05811c07f91e926119a25e5280a90f3/family1 in testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:37,330 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp, totalSize=50.0 K 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 7706a3be75434b6d8135d8f95552cb4f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting f5bc52ea22ed4da5803df141cb4ca036, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting fb87ae45f5aa4a6681997c26a6c1300b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 413e0f9996e54996b69ed0af14b81f54, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting eb7fce9c4fd64c49bf6fb437d454eb60, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 3366fb78748e4a99a4b2401c2db8536f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 2d2982f605474727a4e8a3519924858c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 9c194c04d6bc4d6a845b97f2ff7bf402, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting f23ab9f2005a4222b132fb93352dda7d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:54:37,330 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d80225edd84b480c8ed97599c75bc971_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:54:37,331 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:37,332 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:37,333 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512286c2d0d8285304f718236e10671188a46 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512286c2d0d8285304f718236e10671188a46 2023-03-14 21:54:37,334 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231aebe10d1f26a40cd86de66eb4be424d1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231aebe10d1f26a40cd86de66eb4be424d1 2023-03-14 21:54:37,336 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fb906cfbce54ffa8140bd8c444e0a65 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fb906cfbce54ffa8140bd8c444e0a65 2023-03-14 21:54:37,337 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010381dfb755e9f845a290a6eae0d0827af1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010381dfb755e9f845a290a6eae0d0827af1 2023-03-14 21:54:37,337 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-14 21:54:37,337 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:37,370 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): e05811c07f91e926119a25e5280a90f3#family1#compaction#49 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:38,186 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/8374595407a545dfb9bfca2ae5183ee1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/8374595407a545dfb9bfca2ae5183ee1 2023-03-14 21:54:38,191 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e05811c07f91e926119a25e5280a90f3/family1 of e05811c07f91e926119a25e5280a90f3 into 8374595407a545dfb9bfca2ae5183ee1(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:54:38,191 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:38,191 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678830877329; duration=0sec 2023-03-14 21:54:38,191 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:38,196 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a4c258186f5c44e880a2773434790a4b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205a4c258186f5c44e880a2773434790a4b 2023-03-14 21:54:38,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:38,230 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 first=Optional[r00] last=Optional[r2] 2023-03-14 21:54:38,232 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36}] 2023-03-14 21:54:38,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_] to archive 2023-03-14 21:54:38,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:38,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 for inclusion in e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:38,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/7706a3be75434b6d8135d8f95552cb4f 2023-03-14 21:54:38,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f5bc52ea22ed4da5803df141cb4ca036 2023-03-14 21:54:38,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-14 21:54:38,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:38,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:38,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/fb87ae45f5aa4a6681997c26a6c1300b 2023-03-14 21:54:38,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ohganh03bjh2rsrft16ekoaag2er6d2jnbgkkv0bf2f4dbtevqp30tjk99oejsjb/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 2023-03-14 21:54:38,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/413e0f9996e54996b69ed0af14b81f54 2023-03-14 21:54:38,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/eb7fce9c4fd64c49bf6fb437d454eb60 2023-03-14 21:54:38,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ohganh03bjh2rsrft16ekoaag2er6d2jnbgkkv0bf2f4dbtevqp30tjk99oejsjb/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_ 2023-03-14 21:54:38,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ohganh03bjh2rsrft16ekoaag2er6d2jnbgkkv0bf2f4dbtevqp30tjk99oejsjb/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 into e05811c07f91e926119a25e5280a90f3/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_ - updating store file list. 2023-03-14 21:54:38,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/3366fb78748e4a99a4b2401c2db8536f 2023-03-14 21:54:38,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/2d2982f605474727a4e8a3519924858c 2023-03-14 21:54:38,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_ into e05811c07f91e926119a25e5280a90f3/family1 2023-03-14 21:54:38,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ohganh03bjh2rsrft16ekoaag2er6d2jnbgkkv0bf2f4dbtevqp30tjk99oejsjb/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 into e05811c07f91e926119a25e5280a90f3/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_) 2023-03-14 21:54:38,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/9c194c04d6bc4d6a845b97f2ff7bf402 2023-03-14 21:54:38,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ohganh03bjh2rsrft16ekoaag2er6d2jnbgkkv0bf2f4dbtevqp30tjk99oejsjb/family1/62b67e6a1ddd4b6a83318a9c7ad8ca36 2023-03-14 21:54:38,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:54:38,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:54:38,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/f23ab9f2005a4222b132fb93352dda7d 2023-03-14 21:54:38,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): e05811c07f91e926119a25e5280a90f3/family1 is initiating minor compaction (all files) 2023-03-14 21:54:38,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@22eead0b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:38,252 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e05811c07f91e926119a25e5280a90f3/family1 in testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:54:38,252 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/8374595407a545dfb9bfca2ae5183ee1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp, totalSize=337.0 K 2023-03-14 21:54:38,253 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting e6095325df4049bea1b47d508cb3d7e3, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:54:38,253 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 8374595407a545dfb9bfca2ae5183ee1, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:54:38,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/d80225edd84b480c8ed97599c75bc971_SeqId_44_ 2023-03-14 21:54:38,254 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 65e76c03b6c34192946e2170380aad0c_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845591 2023-03-14 21:54:38,255 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:38,256 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:38,258 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113003609bbc1941431a84a72a60046612f9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113003609bbc1941431a84a72a60046612f9 2023-03-14 21:54:38,259 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120163a54e67fe8b4932a6a089a16fc7514d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120163a54e67fe8b4932a6a089a16fc7514d 2023-03-14 21:54:38,261 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61c6311926642f3bcf95c2c251150b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61c6311926642f3bcf95c2c251150b0 2023-03-14 21:54:38,261 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-14 21:54:38,261 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-14 21:54:38,262 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-14 21:54:38,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-14 21:54:38,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:38,263 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:54:38,265 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8290 sec 2023-03-14 21:54:38,274 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): e05811c07f91e926119a25e5280a90f3#family1#compaction#50 average throughput is 15.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:38,360 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=254 (was 260), OpenFileDescriptor=603 (was 607), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 256), ProcessCount=169 (was 169), AvailableMemoryMB=3326 (was 3260) - AvailableMemoryMB LEAK? - 2023-03-14 21:54:38,372 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=254, OpenFileDescriptor=603, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=169, AvailableMemoryMB=3325 2023-03-14 21:54:38,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:54:38,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:38,376 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:54:38,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-14 21:54:38,377 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:54:38,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:38,378 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:38,379 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb empty. 2023-03-14 21:54:38,379 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:38,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-14 21:54:38,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:38,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:38,787 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:38,788 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => d3dd811eb90158aef21c530fc2c87deb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:38,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing d3dd811eb90158aef21c530fc2c87deb, disabling compactions & flushes 2023-03-14 21:54:39,194 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. after waiting 0 ms 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,194 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,194 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:39,197 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:54:39,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830879197"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830879197"}]},"ts":"1678830879197"} 2023-03-14 21:54:39,199 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:54:39,200 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:54:39,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830879200"}]},"ts":"1678830879200"} 2023-03-14 21:54:39,201 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-14 21:54:39,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, ASSIGN}] 2023-03-14 21:54:39,205 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, ASSIGN 2023-03-14 21:54:39,206 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:39,357 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:39,357 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830879357"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830879357"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830879357"}]},"ts":"1678830879357"} 2023-03-14 21:54:39,359 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:39,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:39,498 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/.tmp/family1/af318e4ff51044cda66a4dcd451883a4 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/af318e4ff51044cda66a4dcd451883a4 2023-03-14 21:54:39,502 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e05811c07f91e926119a25e5280a90f3/family1 of e05811c07f91e926119a25e5280a90f3 into af318e4ff51044cda66a4dcd451883a4(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:54:39,502 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:54:39,502 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1678830878252; duration=1sec 2023-03-14 21:54:39,503 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:39,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d3dd811eb90158aef21c530fc2c87deb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,515 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,516 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:39,516 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:39,517 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d3dd811eb90158aef21c530fc2c87deb columnFamilyName family1 2023-03-14 21:54:39,518 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(310): Store=d3dd811eb90158aef21c530fc2c87deb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:39,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:39,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:39,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d3dd811eb90158aef21c530fc2c87deb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69366699, jitterRate=0.03364436328411102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:39,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:39,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., pid=41, masterSystemTime=1678830879511 2023-03-14 21:54:39,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/8374595407a545dfb9bfca2ae5183ee1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_] to archive 2023-03-14 21:54:39,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:39,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:39,529 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:39,529 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830879529"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830879529"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830879529"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830879529"}]},"ts":"1678830879529"} 2023-03-14 21:54:39,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/e6095325df4049bea1b47d508cb3d7e3 2023-03-14 21:54:39,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/8374595407a545dfb9bfca2ae5183ee1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/8374595407a545dfb9bfca2ae5183ee1 2023-03-14 21:54:39,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-14 21:54:39,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 171 msec 2023-03-14 21:54:39,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/family1/65e76c03b6c34192946e2170380aad0c_SeqId_47_ 2023-03-14 21:54:39,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-14 21:54:39,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, ASSIGN in 328 msec 2023-03-14 21:54:39,534 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:54:39,534 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830879534"}]},"ts":"1678830879534"} 2023-03-14 21:54:39,535 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-14 21:54:39,539 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:54:39,540 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1650 sec 2023-03-14 21:54:40,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-14 21:54:40,483 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-14 21:54:40,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:40,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,504 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:40,504 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:40,504 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:40,504 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:40,505 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,505 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,506 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:40,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:40,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:40,506 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:40,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,507 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:40,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:40,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:40,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:40,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:40,508 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:40,508 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:40,508 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:40,509 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:54:40,926 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130133824c89b6941cdb9580c5baa8dbd43 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130133824c89b6941cdb9580c5baa8dbd43 2023-03-14 21:54:41,330 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6191d96ce00b4e83bc161191eb74d7f6 2023-03-14 21:54:41,336 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6191d96ce00b4e83bc161191eb74d7f6 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6 2023-03-14 21:54:41,340 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-14 21:54:41,341 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 832ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:41,341 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:41,342 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:41,342 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,344 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5dab4e92[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:41,345 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:41,346 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,347 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:41,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:41,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:41,347 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,347 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:41,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:41,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:41,349 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:41,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:41,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:41,351 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:41,351 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:41,351 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:41,351 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,351 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:41,353 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,353 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,353 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:41,353 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,354 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:41,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:41,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:41,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:41,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:41,356 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:41,357 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,357 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,357 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,357 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:41,357 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:41,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:41,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:41,457 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:41,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:41,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:41,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:41,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,464 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:41,464 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:41,464 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:41,464 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:41,465 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,465 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,466 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:41,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:41,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:41,466 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,466 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:41,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:41,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:41,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,467 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:41,467 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:41,467 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:41,467 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:41,468 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:41,468 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:41,468 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:41,468 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:54:41,881 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280a2b363116064a6eb7031877c8258a55 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280a2b363116064a6eb7031877c8258a55 2023-03-14 21:54:41,886 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/3bed32e5dc144e908a6ec780e94f811f 2023-03-14 21:54:41,892 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/3bed32e5dc144e908a6ec780e94f811f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f 2023-03-14 21:54:41,897 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:41,898 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 430ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:41,898 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:41,898 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:41,898 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:41,898 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:41,898 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,898 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:41,898 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:41,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,900 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,900 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,900 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:41,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:41,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4594deb2[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:41,902 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:41,902 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,903 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,903 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:41,903 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:41,903 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:41,903 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,903 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:41,905 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,905 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,905 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,905 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:41,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:41,907 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:41,907 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:41,907 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:41,907 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,907 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:41,912 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,912 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:41,912 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:41,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:41,914 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,923 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:41,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:41,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:41,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:41,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:41,923 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:41,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:41,924 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:41,924 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,024 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:42,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:42,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:42,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:42,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,030 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:42,030 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:42,030 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:42,031 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:42,031 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,031 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,032 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:42,032 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:42,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:42,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,034 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:42,034 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:42,034 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:42,034 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:42,034 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:42,034 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:42,034 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:42,034 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:42,447 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201b4dc65c4796a420ab72263cf6fd543b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4dc65c4796a420ab72263cf6fd543b0 2023-03-14 21:54:42,852 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6c0d97df2dd64746900ba66617c6748e 2023-03-14 21:54:42,858 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6c0d97df2dd64746900ba66617c6748e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e 2023-03-14 21:54:42,862 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e, entries=1, sequenceid=13, filesize=5.0 K 2023-03-14 21:54:42,863 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 829ms, sequenceid=13, compaction requested=true 2023-03-14 21:54:42,863 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:42,863 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:42,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,866 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:42,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,866 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:42,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@692dbe65[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:42,868 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:42,868 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,869 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:42,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:42,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:42,871 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,871 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:42,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:42,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:42,873 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:42,873 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:42,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:42,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:42,873 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,873 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:42,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:42,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:42,875 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,876 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:42,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,879 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:42,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:42,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,880 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:42,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:42,880 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,880 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,880 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,950 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-14 21:54:42,980 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:42,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:42,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:42,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:42,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,987 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:42,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:42,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:42,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:42,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,987 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,988 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:42,988 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:42,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:42,989 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:42,989 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:42,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:42,990 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:42,990 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:42,990 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:43,406 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054d4797f775ac468fa070fd9a54561f08 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054d4797f775ac468fa070fd9a54561f08 2023-03-14 21:54:43,814 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6c68dcc4a8de4001a1213399d4bcabcd 2023-03-14 21:54:43,819 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/6c68dcc4a8de4001a1213399d4bcabcd as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd 2023-03-14 21:54:43,823 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd, entries=1, sequenceid=17, filesize=5.0 K 2023-03-14 21:54:43,824 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 834ms, sequenceid=17, compaction requested=true 2023-03-14 21:54:43,824 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:43,824 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:43,824 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:43,824 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:43,824 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,824 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:43,824 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:43,826 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,826 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:43,826 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,826 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@311efa20[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:43,827 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:43,827 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,828 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,828 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:43,828 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:43,828 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,828 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:43,829 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:43,831 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,831 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,831 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:43,831 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:43,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:43,833 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:43,833 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:43,833 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:43,833 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,833 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:43,834 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,834 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:43,834 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:43,835 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:43,836 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:43,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,838 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:43,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:43,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:43,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:43,838 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,839 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:43,839 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:43,939 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:43,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:43,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:43,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:43,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,946 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:43,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:43,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:43,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:43,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,949 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:43,949 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:43,949 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:43,949 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,949 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:43,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:43,952 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:43,952 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:43,952 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:43,952 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:43,952 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:43,952 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:43,952 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:43,953 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:44,370 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512282671d0551a444e16acccf9b030fa3a05 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512282671d0551a444e16acccf9b030fa3a05 2023-03-14 21:54:44,776 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/1e95702e23d144a386fab960f198828b 2023-03-14 21:54:44,780 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/1e95702e23d144a386fab960f198828b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b 2023-03-14 21:54:44,784 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b, entries=1, sequenceid=21, filesize=5.0 K 2023-03-14 21:54:44,785 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 832ms, sequenceid=21, compaction requested=true 2023-03-14 21:54:44,785 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:44,785 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:44,785 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:44,785 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:44,785 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,785 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:44,785 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:44,790 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,790 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:44,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:44,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:44,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3646fd4f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:44,792 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:44,792 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,793 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,793 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:44,793 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:44,793 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:44,793 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,793 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:44,794 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,794 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:44,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:44,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:44,794 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:44,795 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:44,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:44,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:44,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:44,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:44,797 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:44,797 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:44,797 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:44,797 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,797 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:44,800 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,800 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,800 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:44,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:44,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:44,801 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:44,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,802 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:44,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:44,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,804 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:44,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:44,804 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,805 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:44,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:44,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:44,905 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:44,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:44,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:44,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:44,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,911 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:44,911 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:44,911 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:44,911 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:44,912 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,912 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,913 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:44,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:44,914 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:44,914 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:44,915 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:44,915 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:44,915 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:45,044 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:54:45,330 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313ca7786cb47641e29cc8193f28e24614 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512313ca7786cb47641e29cc8193f28e24614 2023-03-14 21:54:45,336 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/5415c0275a6c4f4c94da9d2326c0124e 2023-03-14 21:54:45,341 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/5415c0275a6c4f4c94da9d2326c0124e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e 2023-03-14 21:54:45,345 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e, entries=1, sequenceid=25, filesize=5.0 K 2023-03-14 21:54:45,346 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 431ms, sequenceid=25, compaction requested=true 2023-03-14 21:54:45,346 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:45,346 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:45,346 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:45,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:45,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:45,347 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:45,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:45,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:45,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,348 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:45,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:45,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:45,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:45,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:45,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55c4c47b[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:45,350 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:45,350 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,352 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,353 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:45,353 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,353 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:45,353 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:45,353 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:45,354 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:45,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:45,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:45,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:45,354 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:45,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:45,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:45,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:45,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:45,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:45,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:45,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:45,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:45,356 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,356 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:45,357 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,357 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,358 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:45,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:45,358 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:45,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,359 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:45,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:45,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:45,361 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,361 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:45,362 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,362 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,362 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:45,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:45,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:45,362 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,362 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:45,462 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:45,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:45,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:45,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:45,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,468 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:45,468 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:45,468 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:45,468 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:45,469 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,469 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,471 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:45,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:45,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:45,471 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:45,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:45,472 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:45,472 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:45,472 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:45,885 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010183692a1408a14ca1a1ccc61ba89b14c6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010183692a1408a14ca1a1ccc61ba89b14c6 2023-03-14 21:54:46,291 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/c9b8c56cd8194557aae0304c5b83ed3d 2023-03-14 21:54:46,296 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/c9b8c56cd8194557aae0304c5b83ed3d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d 2023-03-14 21:54:46,300 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d, entries=1, sequenceid=29, filesize=5.0 K 2023-03-14 21:54:46,301 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 828ms, sequenceid=29, compaction requested=true 2023-03-14 21:54:46,301 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:46,301 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:46,301 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:46,301 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:46,301 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,301 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:46,301 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:46,302 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,302 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:46,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:46,302 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,303 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:46,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:46,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:46,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:46,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:46,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@519e8630[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:46,304 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:46,304 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,305 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,305 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:46,305 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:46,305 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:46,305 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,305 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:46,306 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:46,306 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,306 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:46,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:46,306 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:46,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:46,309 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:46,309 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:46,309 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:46,309 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,309 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:46,314 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,314 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:46,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:46,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,314 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:46,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:46,314 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:46,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:46,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,316 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:46,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:46,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:46,318 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:46,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:46,318 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,319 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:46,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:46,319 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:46,319 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:46,419 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:46,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:46,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:46,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:46,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,425 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:46,425 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:46,425 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:46,425 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:46,426 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,426 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,428 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,428 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:46,428 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:46,428 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:46,429 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:46,429 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:46,429 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:46,842 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f609117f2dfd4d7cbea643c6289f5d6b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f609117f2dfd4d7cbea643c6289f5d6b 2023-03-14 21:54:47,248 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/e5d544f9984341ef97dda3163760a9bb 2023-03-14 21:54:47,253 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/e5d544f9984341ef97dda3163760a9bb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb 2023-03-14 21:54:47,257 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb, entries=1, sequenceid=33, filesize=5.0 K 2023-03-14 21:54:47,258 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 829ms, sequenceid=33, compaction requested=true 2023-03-14 21:54:47,258 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:47,258 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:47,260 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,260 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:47,260 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,260 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:47,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:47,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1797a9d9[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:47,262 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:47,262 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,263 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,263 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:47,263 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:47,263 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,264 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:47,264 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:47,265 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,265 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:47,265 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,265 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:47,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:47,268 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:47,268 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:47,268 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:47,268 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,268 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:47,270 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,270 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:47,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:47,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,270 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:47,271 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:47,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:47,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,272 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:47,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,274 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:47,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:47,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:47,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:47,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:47,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:47,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,275 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:47,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:47,375 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:47,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:47,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:47,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:47,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,381 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:47,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:47,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:47,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:47,382 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,382 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:47,383 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:47,383 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:47,384 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:47,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:47,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,385 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:47,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:47,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:47,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:47,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:47,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:47,385 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:47,385 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:47,385 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:47,385 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:47,798 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078ff2acf574a045738a26bb1a54bfb812 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509078ff2acf574a045738a26bb1a54bfb812 2023-03-14 21:54:48,203 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/16456c260a25441dac2e48db5f31b06d 2023-03-14 21:54:48,208 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/16456c260a25441dac2e48db5f31b06d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d 2023-03-14 21:54:48,212 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d, entries=1, sequenceid=37, filesize=5.0 K 2023-03-14 21:54:48,213 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 828ms, sequenceid=37, compaction requested=true 2023-03-14 21:54:48,213 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:48,213 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:48,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:48,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:48,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:48,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,215 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:48,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:48,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16ba82e1[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:48,216 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:48,216 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,219 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,219 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,219 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:48,220 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:48,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:48,222 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:48,222 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:48,222 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:48,222 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,223 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:48,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:48,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:48,224 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:48,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:48,231 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:48,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:48,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:48,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:48,231 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,232 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:48,232 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,332 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:48,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:48,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:48,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:48,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,339 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:48,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:48,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-14 21:54:48,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-14 21:54:48,340 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,340 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,341 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:48,341 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:48,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:48,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-14 21:54:48,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-14 21:54:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-14 21:54:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:48,343 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:48,343 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. started... 2023-03-14 21:54:48,343 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d3dd811eb90158aef21c530fc2c87deb 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:48,755 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112009a1afa6874a4c379e6c72e144a3944c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112009a1afa6874a4c379e6c72e144a3944c 2023-03-14 21:54:49,161 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/dcc901d5c448439188db597da5fc0c0f 2023-03-14 21:54:49,166 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/dcc901d5c448439188db597da5fc0c0f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f 2023-03-14 21:54:49,170 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f, entries=1, sequenceid=41, filesize=5.0 K 2023-03-14 21:54:49,171 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d3dd811eb90158aef21c530fc2c87deb in 828ms, sequenceid=41, compaction requested=true 2023-03-14 21:54:49,171 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:49,171 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:49,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:49,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:49,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-14 21:54:49,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-14 21:54:49,173 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,173 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:49,173 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,173 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:49,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:49,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-14 21:54:49,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@652e6b51[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:49,174 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-14 21:54:49,174 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,179 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-14 21:54:49,179 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-14 21:54:49,179 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,179 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:49,179 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:49,180 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,180 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-14 21:54:49,180 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:49,180 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-14 21:54:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:49,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:49,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-14 21:54:49,182 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-14 21:54:49,182 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:49,182 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:49,182 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,182 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:49,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:49,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:49,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:49,184 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:49,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:49,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:49,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-14 21:54:49,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:49,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:49,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:49,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:49,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:49,188 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:49,188 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:49,288 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:49,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:49,290 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:49,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:49,293 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:49,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:49,293 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831489293 2023-03-14 21:54:49,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:49,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-14 21:54:49,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:49,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x3b0da786 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:49,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@283516ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:49,303 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,138 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034de6139b2e8a48158175bdd2748ef0c2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601034de6139b2e8a48158175bdd2748ef0c2 2023-03-14 21:54:50,139 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:50,151 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:50,152 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34304, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:50,159 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:50,160 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37292, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:50,171 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3aa8ff4295994315acc1360e79bd4502 first=Optional[r3] last=Optional[r6] 2023-03-14 21:54:50,172 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3aa8ff4295994315acc1360e79bd4502}] 2023-03-14 21:54:50,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3aa8ff4295994315acc1360e79bd4502 for inclusion in d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-14 21:54:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:50,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3aa8ff4295994315acc1360e79bd4502 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ln31jqjhv2m5ars612mcc3apc373ljn4jko0e9s6rgiho8e1fenr0h8c9kgninus/family1/3aa8ff4295994315acc1360e79bd4502 2023-03-14 21:54:50,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ln31jqjhv2m5ars612mcc3apc373ljn4jko0e9s6rgiho8e1fenr0h8c9kgninus/family1/3aa8ff4295994315acc1360e79bd4502 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_ 2023-03-14 21:54:50,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ln31jqjhv2m5ars612mcc3apc373ljn4jko0e9s6rgiho8e1fenr0h8c9kgninus/family1/3aa8ff4295994315acc1360e79bd4502 into d3dd811eb90158aef21c530fc2c87deb/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_ - updating store file list. 2023-03-14 21:54:50,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_ into d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:50,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ln31jqjhv2m5ars612mcc3apc373ljn4jko0e9s6rgiho8e1fenr0h8c9kgninus/family1/3aa8ff4295994315acc1360e79bd4502 into d3dd811eb90158aef21c530fc2c87deb/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_) 2023-03-14 21:54:50,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ln31jqjhv2m5ars612mcc3apc373ljn4jko0e9s6rgiho8e1fenr0h8c9kgninus/family1/3aa8ff4295994315acc1360e79bd4502 2023-03-14 21:54:50,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:54:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:54:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): d3dd811eb90158aef21c530fc2c87deb/family1 is initiating minor compaction 2023-03-14 21:54:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1075d25d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:50,192 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d3dd811eb90158aef21c530fc2c87deb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:50,192 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp, totalSize=50.4 K 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 3bed32e5dc144e908a6ec780e94f811f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6c0d97df2dd64746900ba66617c6748e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6c68dcc4a8de4001a1213399d4bcabcd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 1e95702e23d144a386fab960f198828b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5415c0275a6c4f4c94da9d2326c0124e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:54:50,192 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c9b8c56cd8194557aae0304c5b83ed3d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:54:50,193 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting e5d544f9984341ef97dda3163760a9bb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:54:50,193 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 16456c260a25441dac2e48db5f31b06d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:54:50,193 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting dcc901d5c448439188db597da5fc0c0f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:54:50,193 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 8c6eb3133a9744339d21146412959f75_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:54:50,196 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:50,197 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:50,199 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512282671d0551a444e16acccf9b030fa3a05 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512282671d0551a444e16acccf9b030fa3a05 2023-03-14 21:54:50,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512313ca7786cb47641e29cc8193f28e24614 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512313ca7786cb47641e29cc8193f28e24614 2023-03-14 21:54:50,203 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010183692a1408a14ca1a1ccc61ba89b14c6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010183692a1408a14ca1a1ccc61ba89b14c6 2023-03-14 21:54:50,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f609117f2dfd4d7cbea643c6289f5d6b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f609117f2dfd4d7cbea643c6289f5d6b 2023-03-14 21:54:50,205 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-14 21:54:50,205 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-14 21:54:50,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,235 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): d3dd811eb90158aef21c530fc2c87deb#family1#compaction#61 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:50,275 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/e6216dacf39a477c98a8fd428ef96890 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e6216dacf39a477c98a8fd428ef96890 2023-03-14 21:54:50,280 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d3dd811eb90158aef21c530fc2c87deb/family1 of d3dd811eb90158aef21c530fc2c87deb into e6216dacf39a477c98a8fd428ef96890(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:54:50,280 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:50,280 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678830890192; duration=0sec 2023-03-14 21:54:50,280 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:50,287 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205eb51dc3567184ef7b8bcacd7caca9404 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb51dc3567184ef7b8bcacd7caca9404 2023-03-14 21:54:50,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:50,321 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eed2e2b16eec46f49a65ab17eadd8228 first=Optional[r00] last=Optional[r2] 2023-03-14 21:54:50,324 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eed2e2b16eec46f49a65ab17eadd8228}] 2023-03-14 21:54:50,329 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_] to archive 2023-03-14 21:54:50,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:50,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eed2e2b16eec46f49a65ab17eadd8228 for inclusion in d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:50,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/3bed32e5dc144e908a6ec780e94f811f 2023-03-14 21:54:50,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-14 21:54:50,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:50,334 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c0d97df2dd64746900ba66617c6748e 2023-03-14 21:54:50,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:50,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eed2e2b16eec46f49a65ab17eadd8228 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kgfqso2bslro4ucfosb9rq355ksqg4f4m3m8595neoqo27qam6uk373nfcln9d5l/family1/eed2e2b16eec46f49a65ab17eadd8228 2023-03-14 21:54:50,335 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6c68dcc4a8de4001a1213399d4bcabcd 2023-03-14 21:54:50,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/1e95702e23d144a386fab960f198828b 2023-03-14 21:54:50,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kgfqso2bslro4ucfosb9rq355ksqg4f4m3m8595neoqo27qam6uk373nfcln9d5l/family1/eed2e2b16eec46f49a65ab17eadd8228 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_ 2023-03-14 21:54:50,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kgfqso2bslro4ucfosb9rq355ksqg4f4m3m8595neoqo27qam6uk373nfcln9d5l/family1/eed2e2b16eec46f49a65ab17eadd8228 into d3dd811eb90158aef21c530fc2c87deb/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_ - updating store file list. 2023-03-14 21:54:50,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/5415c0275a6c4f4c94da9d2326c0124e 2023-03-14 21:54:50,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/c9b8c56cd8194557aae0304c5b83ed3d 2023-03-14 21:54:50,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e5d544f9984341ef97dda3163760a9bb 2023-03-14 21:54:50,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_ into d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:50,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kgfqso2bslro4ucfosb9rq355ksqg4f4m3m8595neoqo27qam6uk373nfcln9d5l/family1/eed2e2b16eec46f49a65ab17eadd8228 into d3dd811eb90158aef21c530fc2c87deb/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_) 2023-03-14 21:54:50,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kgfqso2bslro4ucfosb9rq355ksqg4f4m3m8595neoqo27qam6uk373nfcln9d5l/family1/eed2e2b16eec46f49a65ab17eadd8228 2023-03-14 21:54:50,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:54:50,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/16456c260a25441dac2e48db5f31b06d 2023-03-14 21:54:50,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:54:50,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): d3dd811eb90158aef21c530fc2c87deb/family1 is initiating minor compaction (all files) 2023-03-14 21:54:50,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@63fe9f93; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:50,343 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d3dd811eb90158aef21c530fc2c87deb/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:50,343 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e6216dacf39a477c98a8fd428ef96890, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp, totalSize=390.1 K 2023-03-14 21:54:50,344 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6191d96ce00b4e83bc161191eb74d7f6, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:54:50,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/dcc901d5c448439188db597da5fc0c0f 2023-03-14 21:54:50,345 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting e6216dacf39a477c98a8fd428ef96890, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:54:50,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:50,346 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845591 2023-03-14 21:54:50,346 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:50,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/8c6eb3133a9744339d21146412959f75_SeqId_44_ 2023-03-14 21:54:50,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130133824c89b6941cdb9580c5baa8dbd43 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130133824c89b6941cdb9580c5baa8dbd43 2023-03-14 21:54:50,349 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4dc65c4796a420ab72263cf6fd543b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4dc65c4796a420ab72263cf6fd543b0 2023-03-14 21:54:50,350 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054d4797f775ac468fa070fd9a54561f08 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054d4797f775ac468fa070fd9a54561f08 2023-03-14 21:54:50,350 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-14 21:54:50,351 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-14 21:54:50,351 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:50,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:50,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:50,352 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:50,354 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.0620 sec 2023-03-14 21:54:50,364 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): d3dd811eb90158aef21c530fc2c87deb#family1#compaction#62 average throughput is 22.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:54:50,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-14 21:54:50,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:50,841 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-14 21:54:50,842 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:50,842 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-14 21:54:50,844 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-14 21:54:50,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE}] 2023-03-14 21:54:50,847 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE 2023-03-14 21:54:50,848 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:50,848 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830890848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830890848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830890848"}]},"ts":"1678830890848"} 2023-03-14 21:54:50,849 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:51,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d3dd811eb90158aef21c530fc2c87deb, disabling compactions & flushes 2023-03-14 21:54:51,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,582 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/.tmp/family1/2b80e196cedf4cb4afbd5b2a2cb49f7a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/2b80e196cedf4cb4afbd5b2a2cb49f7a 2023-03-14 21:54:51,586 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d3dd811eb90158aef21c530fc2c87deb/family1 of d3dd811eb90158aef21c530fc2c87deb into 2b80e196cedf4cb4afbd5b2a2cb49f7a(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:54:51,586 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:51,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,586 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1678830890343; duration=1sec 2023-03-14 21:54:51,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,586 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:54:51,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. after waiting 0 ms 2023-03-14 21:54:51,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,588 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e6216dacf39a477c98a8fd428ef96890, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_] to archive 2023-03-14 21:54:51,589 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:51,590 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6191d96ce00b4e83bc161191eb74d7f6 2023-03-14 21:54:51,591 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e6216dacf39a477c98a8fd428ef96890 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/e6216dacf39a477c98a8fd428ef96890 2023-03-14 21:54:51,592 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/6f4f9d1797a749e8953d6abc70731bfb_SeqId_47_ 2023-03-14 21:54:51,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-14 21:54:51,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:51,598 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: d3dd811eb90158aef21c530fc2c87deb to self. 2023-03-14 21:54:51,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,599 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=CLOSED 2023-03-14 21:54:51,599 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830891599"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830891599"}]},"ts":"1678830891599"} 2023-03-14 21:54:51,602 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-14 21:54:51,602 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 752 msec 2023-03-14 21:54:51,603 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=true 2023-03-14 21:54:51,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:51,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830891753"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830891753"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830891753"}]},"ts":"1678830891753"} 2023-03-14 21:54:51,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:51,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d3dd811eb90158aef21c530fc2c87deb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:51,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:51,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,911 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,912 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:51,912 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:51,913 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d3dd811eb90158aef21c530fc2c87deb columnFamilyName family1 2023-03-14 21:54:51,918 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/2b80e196cedf4cb4afbd5b2a2cb49f7a 2023-03-14 21:54:51,918 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(310): Store=d3dd811eb90158aef21c530fc2c87deb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:51,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:51,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d3dd811eb90158aef21c530fc2c87deb; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65971530, jitterRate=-0.01694759726524353}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:51,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:51,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., pid=47, masterSystemTime=1678830891906 2023-03-14 21:54:51,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:51,927 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:51,927 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830891927"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830891927"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830891927"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830891927"}]},"ts":"1678830891927"} 2023-03-14 21:54:51,930 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-14 21:54:51,930 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 174 msec 2023-03-14 21:54:51,931 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-14 21:54:51,932 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE in 1.0840 sec 2023-03-14 21:54:51,933 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-14 21:54:51,933 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0880 sec 2023-03-14 21:54:51,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5030 sec 2023-03-14 21:54:51,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-14 21:54:51,937 INFO [Listener at localhost/45975] compactions.TestMobCompactor(278): alter status finished 2023-03-14 21:54:51,938 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:51,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:51,940 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:51,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:51,941 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831491941 2023-03-14 21:54:51,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:51,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-14 21:54:51,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:51,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x1c087ea5 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:51,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c08a3c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:51,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:51,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:51,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:51,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:51,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:52,776 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ceacac4f90e14009825cb33590185f9c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128ceacac4f90e14009825cb33590185f9c 2023-03-14 21:54:52,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:52,789 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:54:52,790 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34310, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:54:52,796 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:54:52,798 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37294, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:54:52,806 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/59b11f026b7745038a564a5a8c2575e3 first=Optional[r05] last=Optional[r8] 2023-03-14 21:54:52,807 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/59b11f026b7745038a564a5a8c2575e3}] 2023-03-14 21:54:52,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/59b11f026b7745038a564a5a8c2575e3 for inclusion in d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:52,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-14 21:54:52,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:54:52,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:52,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/59b11f026b7745038a564a5a8c2575e3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vcb1gsd2slunhg4p97e0g25f065tmo4i4d22f2jhnbbsj6dd48d3ifrug7n0hmrn/family1/59b11f026b7745038a564a5a8c2575e3 2023-03-14 21:54:52,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vcb1gsd2slunhg4p97e0g25f065tmo4i4d22f2jhnbbsj6dd48d3ifrug7n0hmrn/family1/59b11f026b7745038a564a5a8c2575e3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/34d74340659640fda1ae1f937bc327b5_SeqId_53_ 2023-03-14 21:54:52,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vcb1gsd2slunhg4p97e0g25f065tmo4i4d22f2jhnbbsj6dd48d3ifrug7n0hmrn/family1/59b11f026b7745038a564a5a8c2575e3 into d3dd811eb90158aef21c530fc2c87deb/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/34d74340659640fda1ae1f937bc327b5_SeqId_53_ - updating store file list. 2023-03-14 21:54:52,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/34d74340659640fda1ae1f937bc327b5_SeqId_53_ into d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:52,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vcb1gsd2slunhg4p97e0g25f065tmo4i4d22f2jhnbbsj6dd48d3ifrug7n0hmrn/family1/59b11f026b7745038a564a5a8c2575e3 into d3dd811eb90158aef21c530fc2c87deb/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/34d74340659640fda1ae1f937bc327b5_SeqId_53_) 2023-03-14 21:54:52,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vcb1gsd2slunhg4p97e0g25f065tmo4i4d22f2jhnbbsj6dd48d3ifrug7n0hmrn/family1/59b11f026b7745038a564a5a8c2575e3 2023-03-14 21:54:52,825 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:52,826 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:54:52,827 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112009a1afa6874a4c379e6c72e144a3944c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112009a1afa6874a4c379e6c72e144a3944c 2023-03-14 21:54:52,828 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280a2b363116064a6eb7031877c8258a55 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280a2b363116064a6eb7031877c8258a55 2023-03-14 21:54:52,828 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-14 21:54:52,828 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-14 21:54:52,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:52,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:52,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:52,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:52,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:52,830 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:52,831 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 891 msec 2023-03-14 21:54:52,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:54:52,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:53,293 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-14 21:54:53,294 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-14 21:54:53,294 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-14 21:54:53,296 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-14 21:54:53,300 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE}] 2023-03-14 21:54:53,301 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE 2023-03-14 21:54:53,302 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:53,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830893302"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830893302"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830893302"}]},"ts":"1678830893302"} 2023-03-14 21:54:53,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:53,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d3dd811eb90158aef21c530fc2c87deb, disabling compactions & flushes 2023-03-14 21:54:53,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. after waiting 0 ms 2023-03-14 21:54:53,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-14 21:54:53,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:53,463 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: d3dd811eb90158aef21c530fc2c87deb to self. 2023-03-14 21:54:53,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,465 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=CLOSED 2023-03-14 21:54:53,465 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830893465"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830893465"}]},"ts":"1678830893465"} 2023-03-14 21:54:53,467 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-14 21:54:53,467 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 163 msec 2023-03-14 21:54:53,467 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=true 2023-03-14 21:54:53,618 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:53,618 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830893618"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830893618"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830893618"}]},"ts":"1678830893618"} 2023-03-14 21:54:53,620 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:53,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d3dd811eb90158aef21c530fc2c87deb, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:53,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:53,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,776 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,777 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:53,777 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1 2023-03-14 21:54:53,777 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d3dd811eb90158aef21c530fc2c87deb columnFamilyName family1 2023-03-14 21:54:53,782 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/2b80e196cedf4cb4afbd5b2a2cb49f7a 2023-03-14 21:54:53,785 DEBUG [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/family1/34d74340659640fda1ae1f937bc327b5_SeqId_53_ 2023-03-14 21:54:53,786 INFO [StoreOpener-d3dd811eb90158aef21c530fc2c87deb-1] regionserver.HStore(310): Store=d3dd811eb90158aef21c530fc2c87deb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:53,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:54:53,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d3dd811eb90158aef21c530fc2c87deb; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62417811, jitterRate=-0.06990213692188263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:53,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:54:53,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb., pid=53, masterSystemTime=1678830893771 2023-03-14 21:54:53,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:54:53,793 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d3dd811eb90158aef21c530fc2c87deb, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:53,793 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678830893793"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830893793"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830893793"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830893793"}]},"ts":"1678830893793"} 2023-03-14 21:54:53,796 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-14 21:54:53,796 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure d3dd811eb90158aef21c530fc2c87deb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 174 msec 2023-03-14 21:54:53,798 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-14 21:54:53,798 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d3dd811eb90158aef21c530fc2c87deb, REOPEN/MOVE in 496 msec 2023-03-14 21:54:53,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-14 21:54:53,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 503 msec 2023-03-14 21:54:53,802 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 921 msec 2023-03-14 21:54:53,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-14 21:54:53,804 INFO [Listener at localhost/45975] compactions.TestMobCompactor(278): alter status finished 2023-03-14 21:54:53,805 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-14 21:54:53,806 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:53,807 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:53,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:53,807 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831493807 2023-03-14 21:54:53,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:54:53,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-14 21:54:53,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:53,809 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x013d622e to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:54:53,813 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5872df63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-14 21:54:53,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-14 21:54:53,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:54:53,815 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-14 21:54:53,816 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-03-14 21:54:53,865 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=243 (was 254), OpenFileDescriptor=616 (was 603) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 231), ProcessCount=169 (was 169), AvailableMemoryMB=3320 (was 3325) 2023-03-14 21:54:53,876 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=243, OpenFileDescriptor=616, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=169, AvailableMemoryMB=3320 2023-03-14 21:54:53,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-14 21:54:53,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:53,881 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:54:53,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-14 21:54:53,882 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:54:53,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:53,883 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:53,883 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b empty. 2023-03-14 21:54:53,884 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:53,884 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-14 21:54:53,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:54,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:54,292 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-14 21:54:54,293 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 74aa35ec7f44b881e5d675726ee27c9b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:54:54,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:54,700 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:54,700 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 74aa35ec7f44b881e5d675726ee27c9b, disabling compactions & flushes 2023-03-14 21:54:54,700 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:54,700 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:54,701 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. after waiting 0 ms 2023-03-14 21:54:54,701 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:54,701 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:54,701 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:54,704 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:54:54,705 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830894705"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830894705"}]},"ts":"1678830894705"} 2023-03-14 21:54:54,707 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:54:54,707 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:54:54,708 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830894707"}]},"ts":"1678830894707"} 2023-03-14 21:54:54,709 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-14 21:54:54,711 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, ASSIGN}] 2023-03-14 21:54:54,712 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, ASSIGN 2023-03-14 21:54:54,713 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:54:54,864 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:54,864 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830894864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830894864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830894864"}]},"ts":"1678830894864"} 2023-03-14 21:54:54,866 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:54:54,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:55,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:55,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74aa35ec7f44b881e5d675726ee27c9b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:54:55,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:54:55,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,022 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,024 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:54:55,024 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:54:55,024 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74aa35ec7f44b881e5d675726ee27c9b columnFamilyName family1 2023-03-14 21:54:55,025 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(310): Store=74aa35ec7f44b881e5d675726ee27c9b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:54:55,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:54:55,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:54:55,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74aa35ec7f44b881e5d675726ee27c9b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62570570, jitterRate=-0.06762585043907166}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:54:55,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:55,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., pid=57, masterSystemTime=1678830895018 2023-03-14 21:54:55,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:55,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:55,034 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:55,034 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830895034"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830895034"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830895034"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830895034"}]},"ts":"1678830895034"} 2023-03-14 21:54:55,036 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-14 21:54:55,036 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 169 msec 2023-03-14 21:54:55,038 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-14 21:54:55,038 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, ASSIGN in 325 msec 2023-03-14 21:54:55,038 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:54:55,038 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830895038"}]},"ts":"1678830895038"} 2023-03-14 21:54:55,039 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-14 21:54:55,042 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:54:55,043 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1630 sec 2023-03-14 21:54:55,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-14 21:54:55,987 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-14 21:54:56,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:56,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,014 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:56,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:56,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:54:56,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:54:56,014 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,015 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,016 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:56,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:56,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,016 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:54:56,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:56,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:54:56,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:56,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:56,022 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:56,022 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:54:56,023 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:54:56,444 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511305eccd95418794095bcb1c2dbb4ae552c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511305eccd95418794095bcb1c2dbb4ae552c 2023-03-14 21:54:56,848 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/5a6a5a5a41c3460fb38db7685adf812a 2023-03-14 21:54:56,852 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/5a6a5a5a41c3460fb38db7685adf812a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a 2023-03-14 21:54:56,856 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-14 21:54:56,857 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 834ms, sequenceid=5, compaction requested=false 2023-03-14 21:54:56,857 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:56,858 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:56,858 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:56,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:56,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:56,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:54:56,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:54:56,859 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,859 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:56,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:56,859 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,860 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:56,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:56,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:56,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:56,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:54:56,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40a4dec2[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:56,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:54:56,861 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,862 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,862 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:54:56,862 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:54:56,862 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:56,862 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,862 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:56,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:54:56,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:56,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:56,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:56,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:54:56,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:56,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:56,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:56,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:54:56,866 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:54:56,866 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:56,867 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:56,867 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,867 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:56,868 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,868 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,868 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:56,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:56,869 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,870 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:56,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:56,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:56,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:56,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:56,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:56,874 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:56,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:56,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:56,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,875 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,975 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:56,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:56,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:56,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:56,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,981 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:56,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:56,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:54:56,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:54:56,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,981 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,983 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,983 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:56,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:56,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:56,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:54:56,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:54:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:54:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:56,985 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:56,985 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:54:56,985 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:54:57,126 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:54:57,169 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x013d622e to 127.0.0.1:58882 2023-03-14 21:54:57,169 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:57,169 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:57,170 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c087ea5 to 127.0.0.1:58882 2023-03-14 21:54:57,170 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:57,170 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:57,170 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3b0da786 to 127.0.0.1:58882 2023-03-14 21:54:57,170 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:57,170 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:54:57,170 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f59b769 to 127.0.0.1:58882 2023-03-14 21:54:57,170 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:54:57,399 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280b92e606a26842f6a134e1e2faf593b2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280b92e606a26842f6a134e1e2faf593b2 2023-03-14 21:54:57,808 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/dca5aa6726b6445dbefdd2bcd8887144 2023-03-14 21:54:57,813 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/dca5aa6726b6445dbefdd2bcd8887144 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144 2023-03-14 21:54:57,817 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:54:57,818 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 833ms, sequenceid=9, compaction requested=false 2023-03-14 21:54:57,818 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:57,818 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:54:57,818 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:54:57,819 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,819 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:57,820 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,820 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:57,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:54:57,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ad77e6e[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:57,821 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:54:57,821 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,823 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,823 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,823 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:54:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:54:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:57,825 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,825 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:54:57,825 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:57,825 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:57,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:54:57,827 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:54:57,827 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:57,827 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:57,827 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,827 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:57,829 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,829 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,829 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:57,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:57,829 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:57,830 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:57,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,832 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:57,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:57,832 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:57,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-14 21:54:57,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:57,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:57,833 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:57,833 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:57,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:57,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:57,933 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:57,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:57,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:57,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:57,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,939 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:57,939 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:57,939 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:54:57,939 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:54:57,940 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,940 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,947 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:57,947 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:57,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:57,948 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:57,948 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:54:57,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:57,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:54:57,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:57,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:54:57,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:57,949 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:57,949 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:57,949 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:54:57,949 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:58,361 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201a618ea35f01a4010bb2965a481ab5d55 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201a618ea35f01a4010bb2965a481ab5d55 2023-03-14 21:54:58,766 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/37c3247bc8ef4d02b26b10425adeeb21 2023-03-14 21:54:58,770 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/37c3247bc8ef4d02b26b10425adeeb21 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21 2023-03-14 21:54:58,774 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21, entries=1, sequenceid=13, filesize=5.0 K 2023-03-14 21:54:58,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 826ms, sequenceid=13, compaction requested=true 2023-03-14 21:54:58,775 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:58,775 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:58,775 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:58,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:58,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:54:58,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:54:58,777 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,777 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,778 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:58,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:54:58,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2838545f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:58,779 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:54:58,779 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,780 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,780 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:54:58,780 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:54:58,780 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,780 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:58,781 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:58,782 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,782 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:54:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:58,782 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:58,782 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:54:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:58,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:58,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:54:58,785 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:54:58,785 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:58,785 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:58,785 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,785 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:58,787 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,787 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,787 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:58,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,788 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:58,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,789 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:58,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,791 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-14 21:54:58,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:58,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:58,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:58,792 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:58,791 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,792 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:58,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:58,892 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:58,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:58,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:58,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:58,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,898 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:58,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:58,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:54:58,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:54:58,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,898 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,900 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,900 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:58,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:58,900 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:58,900 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,901 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:58,901 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:54:58,901 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:58,901 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:54:58,902 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:58,902 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:58,902 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:58,902 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:54:58,902 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:59,313 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120574b051558e5945f7b04e09156f9d68f3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120574b051558e5945f7b04e09156f9d68f3 2023-03-14 21:54:59,318 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/17fa9db8b7674c44bcff65cf327b4912 2023-03-14 21:54:59,322 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/17fa9db8b7674c44bcff65cf327b4912 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912 2023-03-14 21:54:59,326 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912, entries=1, sequenceid=17, filesize=5.0 K 2023-03-14 21:54:59,326 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 424ms, sequenceid=17, compaction requested=true 2023-03-14 21:54:59,327 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:54:59,327 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:59,327 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:54:59,327 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:54:59,327 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,327 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:54:59,327 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:54:59,331 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,331 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:59,332 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:59,332 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:54:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:59,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:54:59,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3edac6ea[Count = 0] remaining members to acquire global barrier 2023-03-14 21:54:59,333 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:54:59,333 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,334 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,334 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:54:59,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:54:59,334 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:54:59,334 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:54:59,336 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,336 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:59,336 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:59,336 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:54:59,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:54:59,338 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:54:59,338 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:54:59,338 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:54:59,338 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,338 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:54:59,340 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,340 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:54:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,340 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:54:59,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:59,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:59,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:54:59,344 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,345 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:59,348 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:59,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:59,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:54:59,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:59,348 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,349 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,349 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:54:59,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:54:59,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:54:59,349 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:54:59,449 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:54:59,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:54:59,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:54:59,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:54:59,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,455 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:54:59,455 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:59,455 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:54:59,455 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:54:59,455 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,455 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,457 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,457 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:54:59,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:54:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:54:59,457 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:54:59,457 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:54:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:54:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,458 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:54:59,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:54:59,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:54:59,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:54:59,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:54:59,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:54:59,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:54:59,458 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:54:59,458 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:54:59,459 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:54:59,872 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228182bd1c016aa4ae1abcc906b1e957ff1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228182bd1c016aa4ae1abcc906b1e957ff1 2023-03-14 21:55:00,278 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/9513f98b2c9b4e938cb58c49a7f3624e 2023-03-14 21:55:00,282 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/9513f98b2c9b4e938cb58c49a7f3624e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e 2023-03-14 21:55:00,287 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e, entries=1, sequenceid=21, filesize=5.0 K 2023-03-14 21:55:00,288 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 829ms, sequenceid=21, compaction requested=true 2023-03-14 21:55:00,288 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:00,288 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:00,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:00,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:00,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,289 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:00,289 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:00,290 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,290 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:00,291 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,291 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:00,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:00,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@586e4d3d[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:00,292 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:00,292 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,294 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:00,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:00,294 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,294 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:00,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:00,296 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,296 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:00,297 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:00,297 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:00,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:00,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:00,298 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:00,299 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:00,299 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:00,299 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,299 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:00,300 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,300 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,300 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:00,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:00,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:00,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:00,300 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,301 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:00,303 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,304 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:00,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:00,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:00,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:00,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:00,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:00,304 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:00,405 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:00,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:00,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:00,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:00,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,411 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:00,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:00,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:55:00,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:55:00,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,411 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,413 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:00,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:00,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:00,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:00,413 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:00,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:00,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:55:00,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:00,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:55:00,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:00,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:55:00,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:55:00,414 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:00,414 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:00,414 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:55:00,414 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:55:00,827 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ccd5185d2ad4436190ebafa2e40f71f6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ccd5185d2ad4436190ebafa2e40f71f6 2023-03-14 21:55:01,235 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/bf5a3fd226f6414b8f21bbcb30bd941c 2023-03-14 21:55:01,240 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/bf5a3fd226f6414b8f21bbcb30bd941c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c 2023-03-14 21:55:01,245 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c, entries=1, sequenceid=25, filesize=5.0 K 2023-03-14 21:55:01,246 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 832ms, sequenceid=25, compaction requested=true 2023-03-14 21:55:01,246 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:01,246 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:01,253 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,253 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:01,253 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,253 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:01,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:01,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@99b6c50[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:01,254 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:01,254 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,256 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:01,256 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,256 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:01,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:01,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:01,257 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:01,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:01,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:01,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:01,259 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:01,259 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:01,259 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:01,260 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,260 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:01,261 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,261 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:01,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,261 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:01,262 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:01,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,263 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:01,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,268 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,268 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:01,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:01,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:01,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:01,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:01,269 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:01,269 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:01,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:01,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:01,369 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:01,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:01,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:01,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:01,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,376 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:01,376 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:01,376 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:55:01,376 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:55:01,376 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,376 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,380 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:01,380 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:01,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:01,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,382 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:55:01,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:01,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:55:01,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:01,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:55:01,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:55:01,382 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:01,382 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:01,382 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:55:01,382 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:55:01,794 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010147ffeb9ae1dd41b6867cd93751f93588 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010147ffeb9ae1dd41b6867cd93751f93588 2023-03-14 21:55:02,200 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/36fc9df3a6eb4eeb9dd2cb346e16213a 2023-03-14 21:55:02,204 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/36fc9df3a6eb4eeb9dd2cb346e16213a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a 2023-03-14 21:55:02,208 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a, entries=1, sequenceid=29, filesize=5.0 K 2023-03-14 21:55:02,209 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 827ms, sequenceid=29, compaction requested=true 2023-03-14 21:55:02,209 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:02,209 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:02,211 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,211 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:02,211 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,211 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:02,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:02,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:02,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63eb4b90[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:02,212 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:02,212 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,215 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:02,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:02,215 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:02,215 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,215 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:02,216 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:02,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:02,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:02,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:02,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:02,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:02,218 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:02,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:02,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:02,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,218 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:02,220 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,220 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,220 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:02,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:02,220 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,221 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:02,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:02,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:02,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:02,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:02,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,224 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,224 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:02,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:02,224 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:02,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:02,324 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:02,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:02,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:02,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:02,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,331 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:02,331 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:02,331 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:55:02,331 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:55:02,331 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,332 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,333 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,333 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:02,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:02,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:02,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:55:02,336 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:55:02,336 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:02,336 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:02,336 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:55:02,336 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:55:02,757 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bf4681da99264377826715456d1aad56 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103bf4681da99264377826715456d1aad56 2023-03-14 21:55:03,162 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/c14321607a9f4befb663039725881181 2023-03-14 21:55:03,167 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/c14321607a9f4befb663039725881181 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181 2023-03-14 21:55:03,171 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181, entries=1, sequenceid=33, filesize=5.0 K 2023-03-14 21:55:03,172 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 836ms, sequenceid=33, compaction requested=true 2023-03-14 21:55:03,172 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:03,172 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:03,172 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:03,172 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:03,172 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,172 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:03,172 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:03,174 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,174 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:03,174 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,174 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:03,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:03,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b6e3c91[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:03,176 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:03,176 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,177 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,177 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:03,177 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:03,177 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:03,177 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,177 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:03,178 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:03,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,178 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:03,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,178 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:03,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:03,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:03,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:03,181 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:03,181 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:03,181 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:03,181 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:03,181 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,181 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:03,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,184 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:03,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:03,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:03,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,186 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:03,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:03,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:03,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:03,188 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:03,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,288 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:03,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:03,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:03,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:03,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,295 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:03,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:03,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:55:03,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:55:03,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,295 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,296 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:03,296 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:03,297 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:03,297 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:55:03,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:03,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:55:03,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:03,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:55:03,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:55:03,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:03,298 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:03,298 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:55:03,298 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:55:03,712 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090792872d8d952748168766e00af8a09dda to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015090792872d8d952748168766e00af8a09dda 2023-03-14 21:55:04,118 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/d5bff3c67c3f4020a070903367973891 2023-03-14 21:55:04,122 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/d5bff3c67c3f4020a070903367973891 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891 2023-03-14 21:55:04,127 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891, entries=1, sequenceid=37, filesize=5.0 K 2023-03-14 21:55:04,128 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 830ms, sequenceid=37, compaction requested=true 2023-03-14 21:55:04,128 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:04,128 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,129 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:04,129 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:04,131 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,131 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:04,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:04,131 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,131 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:04,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:04,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:04,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21439f64[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:04,133 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:04,133 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,134 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,134 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:04,135 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:04,135 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:04,135 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,135 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:04,136 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:04,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:04,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:04,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:04,138 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:04,139 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:04,139 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:04,139 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,139 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:04,140 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,140 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:04,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,140 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:04,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:04,141 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,142 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,145 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:04,145 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,145 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:04,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:04,145 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:04,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:04,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:04,145 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:04,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:04,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:04,146 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:04,246 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:04,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:04,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:04,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:04,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,253 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:04,253 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:04,253 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-14 21:55:04,253 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-14 21:55:04,253 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,254 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,255 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:04,255 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:04,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:04,255 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:04,255 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:04,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-14 21:55:04,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:04,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-14 21:55:04,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:04,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-14 21:55:04,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:55:04,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:04,257 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:04,257 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. started... 2023-03-14 21:55:04,257 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74aa35ec7f44b881e5d675726ee27c9b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:55:04,677 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511206cd647ceed61403f99bc25990c720f47 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511206cd647ceed61403f99bc25990c720f47 2023-03-14 21:55:05,083 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/6e7a952109c34b6dbcf0833c528b3db1 2023-03-14 21:55:05,087 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/6e7a952109c34b6dbcf0833c528b3db1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1 2023-03-14 21:55:05,091 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1, entries=1, sequenceid=41, filesize=5.0 K 2023-03-14 21:55:05,092 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74aa35ec7f44b881e5d675726ee27c9b in 835ms, sequenceid=41, compaction requested=true 2023-03-14 21:55:05,092 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:05,092 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:05,092 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:55:05,092 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:55:05,092 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,092 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-14 21:55:05,092 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-14 21:55:05,094 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:05,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,094 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:05,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-14 21:55:05,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73e88ef0[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:05,096 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-14 21:55:05,096 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,097 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,097 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,097 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:05,097 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-14 21:55:05,097 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-14 21:55:05,097 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-14 21:55:05,099 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:05,099 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-14 21:55:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:05,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-14 21:55:05,101 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-14 21:55:05,101 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:05,101 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:05,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,102 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:05,103 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,103 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,103 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:05,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:05,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:05,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:05,103 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,104 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:05,107 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:05,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-14 21:55:05,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:05,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:05,107 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:05,208 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:05,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:05,210 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:05,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:05,212 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:05,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:05,212 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831505212 2023-03-14 21:55:05,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:55:05,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-14 21:55:05,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:05,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x5d1ed979 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:05,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3265bead, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:05,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,058 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010362e6600df33a45218a22b383ec105ac6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010362e6600df33a45218a22b383ec105ac6 2023-03-14 21:55:06,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:06,079 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:06,080 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57994, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:06,088 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:06,089 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42094, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:06,098 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ec40709c785a447e8ec822e13f37a13d first=Optional[r3] last=Optional[r6] 2023-03-14 21:55:06,099 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ec40709c785a447e8ec822e13f37a13d}] 2023-03-14 21:55:06,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ec40709c785a447e8ec822e13f37a13d for inclusion in 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:06,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-14 21:55:06,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:55:06,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:06,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ec40709c785a447e8ec822e13f37a13d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__hjmjvsf6rpubg84afdsrqe1p0kbhh99c6ufmgjpnvts48a6qkpfbil7jah3luf6v/family1/ec40709c785a447e8ec822e13f37a13d 2023-03-14 21:55:06,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__hjmjvsf6rpubg84afdsrqe1p0kbhh99c6ufmgjpnvts48a6qkpfbil7jah3luf6v/family1/ec40709c785a447e8ec822e13f37a13d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_ 2023-03-14 21:55:06,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__hjmjvsf6rpubg84afdsrqe1p0kbhh99c6ufmgjpnvts48a6qkpfbil7jah3luf6v/family1/ec40709c785a447e8ec822e13f37a13d into 74aa35ec7f44b881e5d675726ee27c9b/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_ - updating store file list. 2023-03-14 21:55:06,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_ into 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:06,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__hjmjvsf6rpubg84afdsrqe1p0kbhh99c6ufmgjpnvts48a6qkpfbil7jah3luf6v/family1/ec40709c785a447e8ec822e13f37a13d into 74aa35ec7f44b881e5d675726ee27c9b/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_) 2023-03-14 21:55:06,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__hjmjvsf6rpubg84afdsrqe1p0kbhh99c6ufmgjpnvts48a6qkpfbil7jah3luf6v/family1/ec40709c785a447e8ec822e13f37a13d 2023-03-14 21:55:06,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:55:06,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:55:06,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 74aa35ec7f44b881e5d675726ee27c9b/family1 is initiating minor compaction 2023-03-14 21:55:06,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@16b274a0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:06,118 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 74aa35ec7f44b881e5d675726ee27c9b/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:06,118 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp, totalSize=50.3 K 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting dca5aa6726b6445dbefdd2bcd8887144, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 37c3247bc8ef4d02b26b10425adeeb21, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 17fa9db8b7674c44bcff65cf327b4912, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 9513f98b2c9b4e938cb58c49a7f3624e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting bf5a3fd226f6414b8f21bbcb30bd941c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 36fc9df3a6eb4eeb9dd2cb346e16213a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c14321607a9f4befb663039725881181, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d5bff3c67c3f4020a070903367973891, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6e7a952109c34b6dbcf0833c528b3db1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:55:06,119 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:55:06,121 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:06,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:06,124 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228182bd1c016aa4ae1abcc906b1e957ff1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228182bd1c016aa4ae1abcc906b1e957ff1 2023-03-14 21:55:06,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ccd5185d2ad4436190ebafa2e40f71f6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ccd5185d2ad4436190ebafa2e40f71f6 2023-03-14 21:55:06,126 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010147ffeb9ae1dd41b6867cd93751f93588 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010147ffeb9ae1dd41b6867cd93751f93588 2023-03-14 21:55:06,127 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103bf4681da99264377826715456d1aad56 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103bf4681da99264377826715456d1aad56 2023-03-14 21:55:06,127 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-14 21:55:06,127 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:06,160 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74aa35ec7f44b881e5d675726ee27c9b#family1#compaction#73 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:06,982 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/989b162561554aa3a0da46fe2fdb4f6f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/989b162561554aa3a0da46fe2fdb4f6f 2023-03-14 21:55:06,987 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 74aa35ec7f44b881e5d675726ee27c9b/family1 of 74aa35ec7f44b881e5d675726ee27c9b into 989b162561554aa3a0da46fe2fdb4f6f(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:06,987 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:06,987 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678830906118; duration=0sec 2023-03-14 21:55:06,987 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:06,989 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c45e09789a4c4036a6aaf484e08fd6b7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c45e09789a4c4036a6aaf484e08fd6b7 2023-03-14 21:55:06,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:07,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_] to archive 2023-03-14 21:55:07,029 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1852baa78c5943b5ac38ed8c0e5bbe03 first=Optional[r00] last=Optional[r2] 2023-03-14 21:55:07,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:07,030 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1852baa78c5943b5ac38ed8c0e5bbe03}] 2023-03-14 21:55:07,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/dca5aa6726b6445dbefdd2bcd8887144 2023-03-14 21:55:07,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1852baa78c5943b5ac38ed8c0e5bbe03 for inclusion in 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:07,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/37c3247bc8ef4d02b26b10425adeeb21 2023-03-14 21:55:07,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-14 21:55:07,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:55:07,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:07,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/17fa9db8b7674c44bcff65cf327b4912 2023-03-14 21:55:07,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1852baa78c5943b5ac38ed8c0e5bbe03 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rgi1l8semuhonvu5doi0nsnisjiaid9n68cgutetvqadk129msujsmqqcqlcucvr/family1/1852baa78c5943b5ac38ed8c0e5bbe03 2023-03-14 21:55:07,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9513f98b2c9b4e938cb58c49a7f3624e 2023-03-14 21:55:07,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rgi1l8semuhonvu5doi0nsnisjiaid9n68cgutetvqadk129msujsmqqcqlcucvr/family1/1852baa78c5943b5ac38ed8c0e5bbe03 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_ 2023-03-14 21:55:07,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/bf5a3fd226f6414b8f21bbcb30bd941c 2023-03-14 21:55:07,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rgi1l8semuhonvu5doi0nsnisjiaid9n68cgutetvqadk129msujsmqqcqlcucvr/family1/1852baa78c5943b5ac38ed8c0e5bbe03 into 74aa35ec7f44b881e5d675726ee27c9b/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_ - updating store file list. 2023-03-14 21:55:07,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/36fc9df3a6eb4eeb9dd2cb346e16213a 2023-03-14 21:55:07,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/c14321607a9f4befb663039725881181 2023-03-14 21:55:07,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_ into 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:07,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rgi1l8semuhonvu5doi0nsnisjiaid9n68cgutetvqadk129msujsmqqcqlcucvr/family1/1852baa78c5943b5ac38ed8c0e5bbe03 into 74aa35ec7f44b881e5d675726ee27c9b/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_) 2023-03-14 21:55:07,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rgi1l8semuhonvu5doi0nsnisjiaid9n68cgutetvqadk129msujsmqqcqlcucvr/family1/1852baa78c5943b5ac38ed8c0e5bbe03 2023-03-14 21:55:07,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d5bff3c67c3f4020a070903367973891 2023-03-14 21:55:07,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-14 21:55:07,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-14 21:55:07,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 74aa35ec7f44b881e5d675726ee27c9b/family1 is initiating minor compaction (all files) 2023-03-14 21:55:07,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/6e7a952109c34b6dbcf0833c528b3db1 2023-03-14 21:55:07,051 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 74aa35ec7f44b881e5d675726ee27c9b/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:07,051 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/989b162561554aa3a0da46fe2fdb4f6f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp, totalSize=388.1 K 2023-03-14 21:55:07,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@118a150c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:07,052 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 5a6a5a5a41c3460fb38db7685adf812a, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845591 2023-03-14 21:55:07,052 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 989b162561554aa3a0da46fe2fdb4f6f, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245591 2023-03-14 21:55:07,053 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845591 2023-03-14 21:55:07,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:07,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/d940db2dee2949c3a9a93ba50e33eeef_SeqId_44_ 2023-03-14 21:55:07,055 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:07,057 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511305eccd95418794095bcb1c2dbb4ae552c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511305eccd95418794095bcb1c2dbb4ae552c 2023-03-14 21:55:07,058 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201a618ea35f01a4010bb2965a481ab5d55 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201a618ea35f01a4010bb2965a481ab5d55 2023-03-14 21:55:07,059 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120574b051558e5945f7b04e09156f9d68f3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120574b051558e5945f7b04e09156f9d68f3 2023-03-14 21:55:07,059 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-14 21:55:07,059 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-14 21:55:07,060 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:07,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:07,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:07,062 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:07,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8520 sec 2023-03-14 21:55:07,070 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74aa35ec7f44b881e5d675726ee27c9b#family1#compaction#74 average throughput is 29.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:07,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-14 21:55:07,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:07,539 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-14 21:55:07,540 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-14 21:55:07,540 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-14 21:55:07,542 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-14 21:55:07,544 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE}] 2023-03-14 21:55:07,545 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE 2023-03-14 21:55:07,546 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:07,546 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830907546"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830907546"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830907546"}]},"ts":"1678830907546"} 2023-03-14 21:55:07,547 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:07,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:07,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74aa35ec7f44b881e5d675726ee27c9b, disabling compactions & flushes 2023-03-14 21:55:07,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,287 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/.tmp/family1/637bf61f33464e07bdf4a16dd7224d03 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/637bf61f33464e07bdf4a16dd7224d03 2023-03-14 21:55:08,291 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 74aa35ec7f44b881e5d675726ee27c9b/family1 of 74aa35ec7f44b881e5d675726ee27c9b into 637bf61f33464e07bdf4a16dd7224d03(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:08,292 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:08,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,292 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678830907051; duration=1sec 2023-03-14 21:55:08,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,292 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:08,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. after waiting 0 ms 2023-03-14 21:55:08,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,295 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/989b162561554aa3a0da46fe2fdb4f6f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_] to archive 2023-03-14 21:55:08,295 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:08,297 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/5a6a5a5a41c3460fb38db7685adf812a 2023-03-14 21:55:08,298 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/989b162561554aa3a0da46fe2fdb4f6f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/989b162561554aa3a0da46fe2fdb4f6f 2023-03-14 21:55:08,299 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/8c4b3c29c4df4bf98252dc25778e1c2d_SeqId_47_ 2023-03-14 21:55:08,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-14 21:55:08,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:08,304 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 74aa35ec7f44b881e5d675726ee27c9b to self. 2023-03-14 21:55:08,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=CLOSED 2023-03-14 21:55:08,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830908305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830908305"}]},"ts":"1678830908305"} 2023-03-14 21:55:08,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-14 21:55:08,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 759 msec 2023-03-14 21:55:08,308 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=true 2023-03-14 21:55:08,459 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:08,459 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830908458"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830908458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830908458"}]},"ts":"1678830908458"} 2023-03-14 21:55:08,460 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:08,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74aa35ec7f44b881e5d675726ee27c9b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:55:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,616 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,617 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:08,617 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:08,618 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74aa35ec7f44b881e5d675726ee27c9b columnFamilyName family1 2023-03-14 21:55:08,622 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/637bf61f33464e07bdf4a16dd7224d03 2023-03-14 21:55:08,622 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(310): Store=74aa35ec7f44b881e5d675726ee27c9b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:08,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:08,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74aa35ec7f44b881e5d675726ee27c9b; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71594463, jitterRate=0.06684063374996185}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:55:08,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:08,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., pid=63, masterSystemTime=1678830908612 2023-03-14 21:55:08,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:08,629 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:08,630 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830908629"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830908629"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830908629"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830908629"}]},"ts":"1678830908629"} 2023-03-14 21:55:08,632 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-14 21:55:08,632 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 171 msec 2023-03-14 21:55:08,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-14 21:55:08,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE in 1.0880 sec 2023-03-14 21:55:08,635 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-14 21:55:08,635 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0920 sec 2023-03-14 21:55:08,636 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5080 sec 2023-03-14 21:55:08,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-14 21:55:08,638 INFO [Listener at localhost/45975] compactions.TestMobCompactor(278): alter status finished 2023-03-14 21:55:08,639 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:08,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:08,641 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:08,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:08,641 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831508641 2023-03-14 21:55:08,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:55:08,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-14 21:55:08,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:08,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x6d7c4d2a to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:08,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f7bda38, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:08,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:08,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:08,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:08,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:08,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:09,471 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128184619efed2346d5a2e2aee04479e13c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128184619efed2346d5a2e2aee04479e13c 2023-03-14 21:55:09,471 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:09,484 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:09,485 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58008, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:09,490 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:09,492 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:09,499 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae3fb24b95f54d4f99c38b187d3d2897 first=Optional[r05] last=Optional[r8] 2023-03-14 21:55:09,500 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae3fb24b95f54d4f99c38b187d3d2897}] 2023-03-14 21:55:09,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae3fb24b95f54d4f99c38b187d3d2897 for inclusion in 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:09,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-14 21:55:09,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:55:09,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:09,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ae3fb24b95f54d4f99c38b187d3d2897 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__72uk9ascodge0lulupjkejcbtcenchk0ohbjps4ijtrn4tk0cfd0dqqga08ldqtu/family1/ae3fb24b95f54d4f99c38b187d3d2897 2023-03-14 21:55:09,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__72uk9ascodge0lulupjkejcbtcenchk0ohbjps4ijtrn4tk0cfd0dqqga08ldqtu/family1/ae3fb24b95f54d4f99c38b187d3d2897 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9a6393ff54c54ece9a163491727d7673_SeqId_53_ 2023-03-14 21:55:09,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__72uk9ascodge0lulupjkejcbtcenchk0ohbjps4ijtrn4tk0cfd0dqqga08ldqtu/family1/ae3fb24b95f54d4f99c38b187d3d2897 into 74aa35ec7f44b881e5d675726ee27c9b/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9a6393ff54c54ece9a163491727d7673_SeqId_53_ - updating store file list. 2023-03-14 21:55:09,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9a6393ff54c54ece9a163491727d7673_SeqId_53_ into 74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:09,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__72uk9ascodge0lulupjkejcbtcenchk0ohbjps4ijtrn4tk0cfd0dqqga08ldqtu/family1/ae3fb24b95f54d4f99c38b187d3d2897 into 74aa35ec7f44b881e5d675726ee27c9b/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9a6393ff54c54ece9a163491727d7673_SeqId_53_) 2023-03-14 21:55:09,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__72uk9ascodge0lulupjkejcbtcenchk0ohbjps4ijtrn4tk0cfd0dqqga08ldqtu/family1/ae3fb24b95f54d4f99c38b187d3d2897 2023-03-14 21:55:09,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:09,514 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:09,515 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511206cd647ceed61403f99bc25990c720f47 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511206cd647ceed61403f99bc25990c720f47 2023-03-14 21:55:09,516 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280b92e606a26842f6a134e1e2faf593b2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511280b92e606a26842f6a134e1e2faf593b2 2023-03-14 21:55:09,516 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-14 21:55:09,516 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-14 21:55:09,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:09,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:09,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:09,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:09,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:09,518 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:09,519 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 878 msec 2023-03-14 21:55:09,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-14 21:55:09,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:09,978 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-14 21:55:09,979 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-14 21:55:09,979 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-14 21:55:09,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-14 21:55:09,983 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE}] 2023-03-14 21:55:09,983 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE 2023-03-14 21:55:09,984 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:09,984 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830909984"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830909984"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830909984"}]},"ts":"1678830909984"} 2023-03-14 21:55:09,985 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:10,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74aa35ec7f44b881e5d675726ee27c9b, disabling compactions & flushes 2023-03-14 21:55:10,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. after waiting 0 ms 2023-03-14 21:55:10,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-14 21:55:10,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:10,145 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 74aa35ec7f44b881e5d675726ee27c9b to self. 2023-03-14 21:55:10,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,146 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=CLOSED 2023-03-14 21:55:10,147 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830910146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830910146"}]},"ts":"1678830910146"} 2023-03-14 21:55:10,149 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-14 21:55:10,149 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 163 msec 2023-03-14 21:55:10,149 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=true 2023-03-14 21:55:10,300 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:10,300 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830910300"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830910300"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830910300"}]},"ts":"1678830910300"} 2023-03-14 21:55:10,302 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:10,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74aa35ec7f44b881e5d675726ee27c9b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:55:10,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:10,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,457 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,458 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:10,458 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1 2023-03-14 21:55:10,458 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74aa35ec7f44b881e5d675726ee27c9b columnFamilyName family1 2023-03-14 21:55:10,463 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/637bf61f33464e07bdf4a16dd7224d03 2023-03-14 21:55:10,466 DEBUG [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/family1/9a6393ff54c54ece9a163491727d7673_SeqId_53_ 2023-03-14 21:55:10,466 INFO [StoreOpener-74aa35ec7f44b881e5d675726ee27c9b-1] regionserver.HStore(310): Store=74aa35ec7f44b881e5d675726ee27c9b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:10,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:55:10,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74aa35ec7f44b881e5d675726ee27c9b; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63583998, jitterRate=-0.05252459645271301}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:55:10,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:55:10,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b., pid=69, masterSystemTime=1678830910453 2023-03-14 21:55:10,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:55:10,473 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=74aa35ec7f44b881e5d675726ee27c9b, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:10,473 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678830910473"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830910473"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830910473"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830910473"}]},"ts":"1678830910473"} 2023-03-14 21:55:10,476 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-14 21:55:10,476 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 74aa35ec7f44b881e5d675726ee27c9b, server=jenkins-hbase4.apache.org,35143,1678830821197 in 173 msec 2023-03-14 21:55:10,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-14 21:55:10,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=74aa35ec7f44b881e5d675726ee27c9b, REOPEN/MOVE in 493 msec 2023-03-14 21:55:10,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-14 21:55:10,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 498 msec 2023-03-14 21:55:10,480 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 911 msec 2023-03-14 21:55:10,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-14 21:55:10,482 INFO [Listener at localhost/45975] compactions.TestMobCompactor(278): alter status finished 2023-03-14 21:55:10,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-14 21:55:10,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:10,485 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:10,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:10,485 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831510485 2023-03-14 21:55:10,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:55:10,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-14 21:55:10,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:10,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x3eba1225 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:10,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36b1e9da, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-14 21:55:10,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-14 21:55:10,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:10,493 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-14 21:55:10,494 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-03-14 21:55:10,541 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 243), OpenFileDescriptor=613 (was 616), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 210) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=3306 (was 3320) 2023-03-14 21:55:10,554 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=169, AvailableMemoryMB=3305 2023-03-14 21:55:10,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-14 21:55:10,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:10,561 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:55:10,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-14 21:55:10,561 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:55:10,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:10,563 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:10,563 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f empty. 2023-03-14 21:55:10,564 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:10,564 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-14 21:55:10,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:10,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:10,971 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-14 21:55:10,973 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 21741900f842f33a87df74e48f700e4f, NAME => 'testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:11,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 21741900f842f33a87df74e48f700e4f, disabling compactions & flushes 2023-03-14 21:55:11,378 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. after waiting 0 ms 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,378 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,378 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:11,380 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:55:11,381 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678830911381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830911381"}]},"ts":"1678830911381"} 2023-03-14 21:55:11,382 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:55:11,383 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:55:11,383 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830911383"}]},"ts":"1678830911383"} 2023-03-14 21:55:11,384 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-14 21:55:11,388 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, ASSIGN}] 2023-03-14 21:55:11,389 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, ASSIGN 2023-03-14 21:55:11,390 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:11,541 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=21741900f842f33a87df74e48f700e4f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:11,541 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678830911541"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830911541"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830911541"}]},"ts":"1678830911541"} 2023-03-14 21:55:11,543 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 21741900f842f33a87df74e48f700e4f, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:11,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:11,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 21741900f842f33a87df74e48f700e4f, NAME => 'testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:55:11,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-14 21:55:11,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-14 21:55:11,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:11,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,700 INFO [StoreOpener-21741900f842f33a87df74e48f700e4f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,701 DEBUG [StoreOpener-21741900f842f33a87df74e48f700e4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1 2023-03-14 21:55:11,701 DEBUG [StoreOpener-21741900f842f33a87df74e48f700e4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1 2023-03-14 21:55:11,702 INFO [StoreOpener-21741900f842f33a87df74e48f700e4f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21741900f842f33a87df74e48f700e4f columnFamilyName f1 2023-03-14 21:55:11,703 INFO [StoreOpener-21741900f842f33a87df74e48f700e4f-1] regionserver.HStore(310): Store=21741900f842f33a87df74e48f700e4f/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:11,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:11,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:11,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 21741900f842f33a87df74e48f700e4f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67249944, jitterRate=0.0021022558212280273}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:55:11,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:11,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f., pid=73, masterSystemTime=1678830911694 2023-03-14 21:55:11,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:11,710 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=21741900f842f33a87df74e48f700e4f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:11,710 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678830911710"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830911710"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830911710"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830911710"}]},"ts":"1678830911710"} 2023-03-14 21:55:11,712 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-14 21:55:11,712 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 21741900f842f33a87df74e48f700e4f, server=jenkins-hbase4.apache.org,35143,1678830821197 in 168 msec 2023-03-14 21:55:11,713 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-14 21:55:11,713 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, ASSIGN in 324 msec 2023-03-14 21:55:11,714 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:55:11,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830911714"}]},"ts":"1678830911714"} 2023-03-14 21:55:11,715 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-14 21:55:11,716 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:55:11,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1580 sec 2023-03-14 21:55:12,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-14 21:55:12,667 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-14 21:55:12,667 DEBUG [Listener at localhost/45975] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-14 21:55:12,670 INFO [Listener at localhost/45975] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-14 21:55:12,673 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-14 21:55:12,674 INFO [Listener at localhost/45975] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-14 21:55:12,674 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-14 21:55:12,678 INFO [Listener at localhost/45975] regionserver.HRegion(2745): Flushing 21741900f842f33a87df74e48f700e4f 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-14 21:55:12,886 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-14 21:55:13,090 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314c2db1bf6a8e14705a53552cf85e56961 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230314c2db1bf6a8e14705a53552cf85e56961 2023-03-14 21:55:13,095 INFO [Listener at localhost/45975] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp/f1/25f0d336866d41ef928e006adfcb9be0 2023-03-14 21:55:13,100 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp/f1/25f0d336866d41ef928e006adfcb9be0 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0 2023-03-14 21:55:13,104 INFO [Listener at localhost/45975] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0, entries=1, sequenceid=5, filesize=4.9 K 2023-03-14 21:55:13,105 INFO [Listener at localhost/45975] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 21741900f842f33a87df74e48f700e4f in 427ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:13,106 DEBUG [Listener at localhost/45975] regionserver.HRegion(2446): Flush status journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:13,108 INFO [Listener at localhost/45975] regionserver.HRegion(2745): Flushing 21741900f842f33a87df74e48f700e4f 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-14 21:55:13,526 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303149d6422fed7054894a4abfa8d9938255c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303149d6422fed7054894a4abfa8d9938255c 2023-03-14 21:55:13,845 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:55:13,931 INFO [Listener at localhost/45975] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp/f1/4e63ff3dcd114d9dabb1512e4d6f99c0 2023-03-14 21:55:13,935 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp/f1/4e63ff3dcd114d9dabb1512e4d6f99c0 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0 2023-03-14 21:55:13,939 INFO [Listener at localhost/45975] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0, entries=1, sequenceid=9, filesize=4.9 K 2023-03-14 21:55:13,940 INFO [Listener at localhost/45975] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 21741900f842f33a87df74e48f700e4f in 832ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:13,941 DEBUG [Listener at localhost/45975] regionserver.HRegion(2446): Flush status journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:13,943 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-14 21:55:13,944 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-14 21:55:13,945 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-14 21:55:13,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:13,945 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831513945 2023-03-14 21:55:13,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:55:13,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-14 21:55:13,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-14 21:55:13,946 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x648a08e4 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:13,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5dccc296, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:13,951 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230314 for table='testGetAfterCompaction' and column='f1' 2023-03-14 21:55:13,994 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:13,994 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,045 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,045 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,096 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,096 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,147 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,147 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,198 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,198 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,249 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,249 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,300 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,300 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,351 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,351 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,402 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,402 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,453 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,453 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,504 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,504 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,555 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,555 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,606 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,606 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,656 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,657 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,707 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,707 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,758 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,758 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,774 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303145c7774fc689c4533b249d6795a76e792 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303145c7774fc689c4533b249d6795a76e792 2023-03-14 21:55:14,775 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testGetAfterCompaction' column='f1' 2023-03-14 21:55:14,788 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:14,789 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48490, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:14,794 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:14,796 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35084, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:14,804 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/f1/ecb45aaa96364991839e0e95fe68a3ff first=Optional[r1] last=Optional[r2] 2023-03-14 21:55:14,805 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{f1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/f1/ecb45aaa96364991839e0e95fe68a3ff}] 2023-03-14 21:55:14,809 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,809 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/f1/ecb45aaa96364991839e0e95fe68a3ff for inclusion in 21741900f842f33a87df74e48f700e4f/f1 2023-03-14 21:55:14,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-14 21:55:14,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:55:14,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:14,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/f1/ecb45aaa96364991839e0e95fe68a3ff to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testGetAfterCompaction__to28gl3pi5ekbrb2ffc8rfqg9ibn93eq6924t7nail5k371a3je1r8jd3iqlhvpp/f1/ecb45aaa96364991839e0e95fe68a3ff 2023-03-14 21:55:14,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testGetAfterCompaction__to28gl3pi5ekbrb2ffc8rfqg9ibn93eq6924t7nail5k371a3je1r8jd3iqlhvpp/f1/ecb45aaa96364991839e0e95fe68a3ff as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_ 2023-03-14 21:55:14,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testGetAfterCompaction__to28gl3pi5ekbrb2ffc8rfqg9ibn93eq6924t7nail5k371a3je1r8jd3iqlhvpp/f1/ecb45aaa96364991839e0e95fe68a3ff into 21741900f842f33a87df74e48f700e4f/f1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_ - updating store file list. 2023-03-14 21:55:14,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_ into 21741900f842f33a87df74e48f700e4f/f1 2023-03-14 21:55:14,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testGetAfterCompaction__to28gl3pi5ekbrb2ffc8rfqg9ibn93eq6924t7nail5k371a3je1r8jd3iqlhvpp/f1/ecb45aaa96364991839e0e95fe68a3ff into 21741900f842f33a87df74e48f700e4f/f1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_) 2023-03-14 21:55:14,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testGetAfterCompaction__to28gl3pi5ekbrb2ffc8rfqg9ibn93eq6924t7nail5k371a3je1r8jd3iqlhvpp/f1/ecb45aaa96364991839e0e95fe68a3ff 2023-03-14 21:55:14,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): 21741900f842f33a87df74e48f700e4f/f1 is initiating minor compaction 2023-03-14 21:55:14,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@17b29980; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:14,820 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 21741900f842f33a87df74e48f700e4f/f1 in testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:14,820 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp, totalSize=10.0 K 2023-03-14 21:55:14,820 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 4e63ff3dcd114d9dabb1512e4d6f99c0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:55:14,820 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 6ec93b8475034a63b83aedc4ca66505b_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:55:14,822 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testGetAfterCompaction' column='f1' 2023-03-14 21:55:14,823 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:14,824 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303149d6422fed7054894a4abfa8d9938255c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303149d6422fed7054894a4abfa8d9938255c 2023-03-14 21:55:14,826 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230314c2db1bf6a8e14705a53552cf85e56961 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230314c2db1bf6a8e14705a53552cf85e56961 2023-03-14 21:55:14,826 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-14 21:55:14,826 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230314' 2023-03-14 21:55:14,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230314 for table='testGetAfterCompaction' and column='f1' 2023-03-14 21:55:14,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-14 21:55:14,828 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 21741900f842f33a87df74e48f700e4f#f1#compaction#77 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:14,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:55:14,829 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-14 21:55:14,830 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 886 msec 2023-03-14 21:55:14,860 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,860 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,860 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:14,865 DEBUG [Listener at localhost/45975] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-14 21:55:14,868 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48506, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-14 21:55:14,868 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35143] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:14,869 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-14 21:55:14,869 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-14 21:55:14,869 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35143] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-14 21:55:14,869 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35143] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. because compaction request was cancelled 2023-03-14 21:55:14,921 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,921 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,921 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:14,972 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,972 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:14,972 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,023 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,023 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,023 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,074 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,074 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,074 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,125 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,125 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,126 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,176 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,177 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,177 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,228 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,228 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,228 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,279 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,279 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,279 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,330 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,330 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,330 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,381 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,381 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,381 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,432 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,432 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,432 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,483 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,483 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,483 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,534 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,534 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,534 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,585 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,585 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,585 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,636 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,636 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,636 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,642 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/.tmp/f1/d14b67ff3f0a4ee0af03788090475e4c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/d14b67ff3f0a4ee0af03788090475e4c 2023-03-14 21:55:15,647 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 21741900f842f33a87df74e48f700e4f/f1 of 21741900f842f33a87df74e48f700e4f into d14b67ff3f0a4ee0af03788090475e4c(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:15,647 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:15,647 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678830914820; duration=0sec 2023-03-14 21:55:15,647 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:15,687 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,687 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913931; access_time=1678830913110; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,687 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678830914770; access_time=1678830913962; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-14 21:55:15,687 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/d14b67ff3f0a4ee0af03788090475e4c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678830915638; access_time=1678830914825; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_] to archive 2023-03-14 21:55:15,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:15,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/4e63ff3dcd114d9dabb1512e4d6f99c0 2023-03-14 21:55:15,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/6ec93b8475034a63b83aedc4ca66505b_SeqId_12_ 2023-03-14 21:55:15,738 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678830913095; access_time=1678830912681; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,738 INFO [Listener at localhost/45975] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/d14b67ff3f0a4ee0af03788090475e4c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678830915638; access_time=1678830914825; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-14 21:55:15,853 INFO [Listener at localhost/45975] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-14 21:55:15,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-14 21:55:15,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:15,865 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830915865"}]},"ts":"1678830915865"} 2023-03-14 21:55:15,867 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-14 21:55:15,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-14 21:55:15,868 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-14 21:55:15,869 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, UNASSIGN}] 2023-03-14 21:55:15,871 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, UNASSIGN 2023-03-14 21:55:15,871 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=21741900f842f33a87df74e48f700e4f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:15,871 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678830915871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830915871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830915871"}]},"ts":"1678830915871"} 2023-03-14 21:55:15,872 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 21741900f842f33a87df74e48f700e4f, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:15,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-14 21:55:16,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:16,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 21741900f842f33a87df74e48f700e4f, disabling compactions & flushes 2023-03-14 21:55:16,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:16,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:16,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. after waiting 0 ms 2023-03-14 21:55:16,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:16,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-14 21:55:16,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-14 21:55:16,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f. 2023-03-14 21:55:16,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 21741900f842f33a87df74e48f700e4f: 2023-03-14 21:55:16,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:16,031 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=21741900f842f33a87df74e48f700e4f, regionState=CLOSED 2023-03-14 21:55:16,031 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678830916031"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830916031"}]},"ts":"1678830916031"} 2023-03-14 21:55:16,034 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-14 21:55:16,034 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 21741900f842f33a87df74e48f700e4f, server=jenkins-hbase4.apache.org,35143,1678830821197 in 160 msec 2023-03-14 21:55:16,035 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-14 21:55:16,035 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=21741900f842f33a87df74e48f700e4f, UNASSIGN in 165 msec 2023-03-14 21:55:16,035 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830916035"}]},"ts":"1678830916035"} 2023-03-14 21:55:16,036 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-14 21:55:16,042 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-14 21:55:16,043 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-03-14 21:55:16,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-14 21:55:16,169 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-14 21:55:16,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-14 21:55:16,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,181 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,182 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-14 21:55:16,185 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:16,187 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1, FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/recovered.edits] 2023-03-14 21:55:16,190 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/25f0d336866d41ef928e006adfcb9be0 2023-03-14 21:55:16,191 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/d14b67ff3f0a4ee0af03788090475e4c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/f1/d14b67ff3f0a4ee0af03788090475e4c 2023-03-14 21:55:16,194 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/recovered.edits/15.seqid to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f/recovered.edits/15.seqid 2023-03-14 21:55:16,195 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testGetAfterCompaction/21741900f842f33a87df74e48f700e4f 2023-03-14 21:55:16,195 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-14 21:55:16,195 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-14 21:55:16,197 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-14 21:55:16,200 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303145c7774fc689c4533b249d6795a76e792 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303145c7774fc689c4533b249d6795a76e792 2023-03-14 21:55:16,200 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-14 21:55:16,202 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,231 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-14 21:55:16,233 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-14 21:55:16,235 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,235 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-14 21:55:16,235 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678830916235"}]},"ts":"9223372036854775807"} 2023-03-14 21:55:16,236 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-14 21:55:16,236 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 21741900f842f33a87df74e48f700e4f, NAME => 'testGetAfterCompaction,,1678830910558.21741900f842f33a87df74e48f700e4f.', STARTKEY => '', ENDKEY => ''}] 2023-03-14 21:55:16,236 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-14 21:55:16,236 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678830916236"}]},"ts":"9223372036854775807"} 2023-03-14 21:55:16,238 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-14 21:55:16,240 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-14 21:55:16,241 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 65 msec 2023-03-14 21:55:16,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-14 21:55:16,285 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-14 21:55:16,299 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=246 (was 240) - Thread LEAK? -, OpenFileDescriptor=616 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=216 (was 225), ProcessCount=169 (was 169), AvailableMemoryMB=3299 (was 3305) 2023-03-14 21:55:16,311 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=246, OpenFileDescriptor=616, MaxFileDescriptor=60000, SystemLoadAverage=216, ProcessCount=169, AvailableMemoryMB=3299 2023-03-14 21:55:16,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-14 21:55:16,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-14 21:55:16,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:55:16,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-14 21:55:16,316 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:55:16,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-14 21:55:16,318 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:16,318 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:16,318 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 empty. 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 empty. 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff empty. 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:16,319 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:16,319 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-14 21:55:16,333 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-14 21:55:16,335 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => f16657fdf561e3e0eb764d4e08e94891, NAME => 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:16,335 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 068a37104fe03c938dbf739e3e0c8aff, NAME => 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:16,335 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => df07f4658e3072f4f5757b3e959f3960, NAME => 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing f16657fdf561e3e0eb764d4e08e94891, disabling compactions & flushes 2023-03-14 21:55:16,349 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. after waiting 0 ms 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:16,349 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:16,349 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:16,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-14 21:55:16,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-14 21:55:16,750 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:16,751 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 068a37104fe03c938dbf739e3e0c8aff, disabling compactions & flushes 2023-03-14 21:55:16,751 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:16,751 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:16,751 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. after waiting 0 ms 2023-03-14 21:55:16,751 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:16,751 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:16,751 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing df07f4658e3072f4f5757b3e959f3960, disabling compactions & flushes 2023-03-14 21:55:16,754 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. after waiting 0 ms 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:16,754 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:16,754 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:16,756 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:55:16,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830916757"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830916757"}]},"ts":"1678830916757"} 2023-03-14 21:55:16,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830916757"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830916757"}]},"ts":"1678830916757"} 2023-03-14 21:55:16,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830916757"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830916757"}]},"ts":"1678830916757"} 2023-03-14 21:55:16,759 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-14 21:55:16,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:55:16,760 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830916759"}]},"ts":"1678830916759"} 2023-03-14 21:55:16,761 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-14 21:55:16,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN}] 2023-03-14 21:55:16,766 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN 2023-03-14 21:55:16,766 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN 2023-03-14 21:55:16,766 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN 2023-03-14 21:55:16,766 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:16,766 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:16,766 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:16,918 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:16,918 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:16,918 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:16,918 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830916918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830916918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830916918"}]},"ts":"1678830916918"} 2023-03-14 21:55:16,918 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830916918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830916918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830916918"}]},"ts":"1678830916918"} 2023-03-14 21:55:16,918 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830916918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830916918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830916918"}]},"ts":"1678830916918"} 2023-03-14 21:55:16,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-14 21:55:16,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=81, state=RUNNABLE; OpenRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:16,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:16,923 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:17,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:17,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f16657fdf561e3e0eb764d4e08e94891, NAME => 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.', STARTKEY => '2', ENDKEY => ''} 2023-03-14 21:55:17,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:17,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,077 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,078 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:17,078 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:17,079 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f16657fdf561e3e0eb764d4e08e94891 columnFamilyName family1 2023-03-14 21:55:17,080 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(310): Store=f16657fdf561e3e0eb764d4e08e94891/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,080 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,081 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2 2023-03-14 21:55:17,081 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2 2023-03-14 21:55:17,082 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f16657fdf561e3e0eb764d4e08e94891 columnFamilyName family2 2023-03-14 21:55:17,083 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(310): Store=f16657fdf561e3e0eb764d4e08e94891/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:17,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:17,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:17,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f16657fdf561e3e0eb764d4e08e94891; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68966689, jitterRate=0.027683749794960022}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:17,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:17,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., pid=85, masterSystemTime=1678830917073 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:17,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:17,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 068a37104fe03c938dbf739e3e0c8aff, NAME => 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.', STARTKEY => '1', ENDKEY => '2'} 2023-03-14 21:55:17,091 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:17,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830917091"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830917091"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830917091"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830917091"}]},"ts":"1678830917091"} 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,092 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,093 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:17,093 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:17,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-03-14 21:55:17,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197 in 171 msec 2023-03-14 21:55:17,094 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a37104fe03c938dbf739e3e0c8aff columnFamilyName family1 2023-03-14 21:55:17,095 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN in 330 msec 2023-03-14 21:55:17,095 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(310): Store=068a37104fe03c938dbf739e3e0c8aff/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,095 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,096 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2 2023-03-14 21:55:17,096 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2 2023-03-14 21:55:17,096 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a37104fe03c938dbf739e3e0c8aff columnFamilyName family2 2023-03-14 21:55:17,097 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(310): Store=068a37104fe03c938dbf739e3e0c8aff/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:17,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:17,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:17,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 068a37104fe03c938dbf739e3e0c8aff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73540919, jitterRate=0.09584508836269379}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:17,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:17,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., pid=83, masterSystemTime=1678830917073 2023-03-14 21:55:17,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:17,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:17,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:17,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => df07f4658e3072f4f5757b3e959f3960, NAME => 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.', STARTKEY => '', ENDKEY => '1'} 2023-03-14 21:55:17,105 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:17,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:17,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,105 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830917105"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830917105"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830917105"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830917105"}]},"ts":"1678830917105"} 2023-03-14 21:55:17,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,106 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,107 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:17,107 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:17,107 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=81 2023-03-14 21:55:17,107 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=81, state=SUCCESS; OpenRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197 in 186 msec 2023-03-14 21:55:17,108 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df07f4658e3072f4f5757b3e959f3960 columnFamilyName family1 2023-03-14 21:55:17,108 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN in 344 msec 2023-03-14 21:55:17,108 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(310): Store=df07f4658e3072f4f5757b3e959f3960/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,109 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,110 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2 2023-03-14 21:55:17,110 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2 2023-03-14 21:55:17,111 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df07f4658e3072f4f5757b3e959f3960 columnFamilyName family2 2023-03-14 21:55:17,112 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(310): Store=df07f4658e3072f4f5757b3e959f3960/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:17,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:17,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:17,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:17,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened df07f4658e3072f4f5757b3e959f3960; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72387922, jitterRate=0.07866409420967102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:17,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:17,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., pid=84, masterSystemTime=1678830917073 2023-03-14 21:55:17,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:17,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:17,120 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:17,120 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830917119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830917119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830917119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830917119"}]},"ts":"1678830917119"} 2023-03-14 21:55:17,122 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=80 2023-03-14 21:55:17,122 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197 in 201 msec 2023-03-14 21:55:17,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-14 21:55:17,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN in 359 msec 2023-03-14 21:55:17,123 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:55:17,123 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830917123"}]},"ts":"1678830917123"} 2023-03-14 21:55:17,124 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-14 21:55:17,127 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:55:17,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 814 msec 2023-03-14 21:55:17,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-14 21:55:17,421 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-14 21:55:17,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:17,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:17,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:17,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:17,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-14 21:55:17,435 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-14 21:55:17,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:17,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-14 21:55:17,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-14 21:55:17,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:17,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:17,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:17,437 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:17,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:17,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:17,437 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:17,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:17,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-14 21:55:17,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-14 21:55:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-14 21:55:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-14 21:55:17,438 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:17,438 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:17,438 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:17,438 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:17,442 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:17,443 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:17,443 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:17,443 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df07f4658e3072f4f5757b3e959f3960 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:17,443 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 068a37104fe03c938dbf739e3e0c8aff 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:17,443 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:17,443 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f16657fdf561e3e0eb764d4e08e94891 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:17,856 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a 2023-03-14 21:55:17,857 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 2023-03-14 21:55:17,858 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 2023-03-14 21:55:17,865 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/d9e4054fc8664bc08b5a5206e1eeaab9 2023-03-14 21:55:17,879 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53 2023-03-14 21:55:18,266 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/104d41f04ac64b5d8d71fcd11b80ee11 2023-03-14 21:55:18,266 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/5a71bf3ad2744ecebe89e0ad69ab75ed 2023-03-14 21:55:18,284 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/4dfcc1ed30054965a87ed75c1e9bad70 2023-03-14 21:55:18,288 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/d9e4054fc8664bc08b5a5206e1eeaab9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9 2023-03-14 21:55:18,291 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:18,292 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/4dfcc1ed30054965a87ed75c1e9bad70 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70 2023-03-14 21:55:18,296 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:18,296 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 068a37104fe03c938dbf739e3e0c8aff in 853ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:18,297 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-14 21:55:18,297 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:18,297 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:18,297 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:18,686 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45 2023-03-14 21:55:18,689 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e 2023-03-14 21:55:18,691 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/edd24bca9b49439b9bd49f6f9b0743af 2023-03-14 21:55:18,695 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/104d41f04ac64b5d8d71fcd11b80ee11 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11 2023-03-14 21:55:18,699 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:18,699 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/edd24bca9b49439b9bd49f6f9b0743af as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af 2023-03-14 21:55:18,703 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:18,703 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for df07f4658e3072f4f5757b3e959f3960 in 1260ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:18,704 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:18,704 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:18,704 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:19,094 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/e8d4a324c2e54e328bd67e2b2377a1ea 2023-03-14 21:55:19,098 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/5a71bf3ad2744ecebe89e0ad69ab75ed as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed 2023-03-14 21:55:19,101 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:19,102 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/e8d4a324c2e54e328bd67e2b2377a1ea as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea 2023-03-14 21:55:19,105 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:19,106 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f16657fdf561e3e0eb764d4e08e94891 in 1663ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:19,106 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:19,106 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:19,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:19,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:19,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-14 21:55:19,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-14 21:55:19,108 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,108 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:19,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:19,108 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,108 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:19,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:19,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:19,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:19,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-14 21:55:19,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a66fcd6[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:19,109 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-14 21:55:19,109 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,110 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-14 21:55:19,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-14 21:55:19,110 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,111 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:19,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:19,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,113 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-14 21:55:19,113 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:19,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:19,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:19,113 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-14 21:55:19,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:19,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:19,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:19,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:19,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-14 21:55:19,115 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-14 21:55:19,115 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:19,115 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:19,115 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-14 21:55:19,115 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:19,117 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:19,117 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:19,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:19,117 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:19,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,118 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:19,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:19,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:19,120 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:19,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-14 21:55:19,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:19,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:19,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:19,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-14 21:55:19,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,121 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:19,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:19,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:19,121 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:19,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:19,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,221 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:19,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:19,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:19,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:19,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:19,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:19,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:19,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-14 21:55:19,227 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-14 21:55:19,227 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:19,227 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-14 21:55:19,227 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-14 21:55:19,228 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,228 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:19,229 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,229 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:19,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:19,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:19,229 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:19,229 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:19,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:19,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:19,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:19,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:19,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-14 21:55:19,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-14 21:55:19,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-14 21:55:19,230 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-14 21:55:19,230 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:19,230 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:19,231 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:19,231 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:19,231 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df07f4658e3072f4f5757b3e959f3960 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:19,231 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 068a37104fe03c938dbf739e3e0c8aff 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:19,231 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f16657fdf561e3e0eb764d4e08e94891 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:19,646 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 2023-03-14 21:55:19,650 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 2023-03-14 21:55:19,650 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 2023-03-14 21:55:19,655 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/613d0296f4cb4dfcb66cb3b44c709ce8 2023-03-14 21:55:19,666 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85 2023-03-14 21:55:20,052 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/c106356df3774a0da464e118374ba796 2023-03-14 21:55:20,055 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/e92e43a4246c4a4e8dae31c701c4de2f 2023-03-14 21:55:20,072 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/5744c93167d04cc8b606b26ae895bfc5 2023-03-14 21:55:20,074 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8 2023-03-14 21:55:20,076 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c 2023-03-14 21:55:20,077 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/613d0296f4cb4dfcb66cb3b44c709ce8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8 2023-03-14 21:55:20,081 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:20,081 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/5744c93167d04cc8b606b26ae895bfc5 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5 2023-03-14 21:55:20,083 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/a881deaf5aee415990f4d1bfab3bd1c8 2023-03-14 21:55:20,085 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:20,085 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/6de3f715832e4389b1a7cc6d9dc660c4 2023-03-14 21:55:20,086 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 068a37104fe03c938dbf739e3e0c8aff in 855ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:20,086 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:20,086 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:20,086 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:20,088 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/c106356df3774a0da464e118374ba796 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796 2023-03-14 21:55:20,089 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/e92e43a4246c4a4e8dae31c701c4de2f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f 2023-03-14 21:55:20,092 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:20,092 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:20,093 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/a881deaf5aee415990f4d1bfab3bd1c8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8 2023-03-14 21:55:20,093 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/6de3f715832e4389b1a7cc6d9dc660c4 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4 2023-03-14 21:55:20,097 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:20,097 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:20,097 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f16657fdf561e3e0eb764d4e08e94891 in 866ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:20,097 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for df07f4658e3072f4f5757b3e959f3960 in 866ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:20,097 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:20,097 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:20,097 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:20,097 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-14 21:55:20,098 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-14 21:55:20,100 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,100 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:20,101 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,101 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:20,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:20,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-14 21:55:20,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3096cbbf[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:20,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-14 21:55:20,102 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,103 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,103 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-14 21:55:20,103 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-14 21:55:20,103 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:20,103 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,103 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:20,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-14 21:55:20,105 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,105 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-14 21:55:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:20,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:20,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:20,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:20,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-14 21:55:20,107 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-14 21:55:20,107 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:20,107 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:20,107 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-14 21:55:20,107 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:20,109 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:20,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:20,109 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,110 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:20,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:20,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:20,113 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:20,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:20,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:20,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-14 21:55:20,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:20,113 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,114 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,114 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:20,114 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:20,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:20,114 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:20,114 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:20,214 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:20,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:20,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:20,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:20,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:20,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:20,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:20,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-14 21:55:20,223 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-14 21:55:20,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:20,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-14 21:55:20,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-14 21:55:20,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:20,225 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:20,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:20,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:20,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:20,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:20,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-14 21:55:20,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-14 21:55:20,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-14 21:55:20,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-14 21:55:20,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:20,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:20,232 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df07f4658e3072f4f5757b3e959f3960 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:20,232 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:20,232 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:20,232 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 068a37104fe03c938dbf739e3e0c8aff 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:20,232 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f16657fdf561e3e0eb764d4e08e94891 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:20,659 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 2023-03-14 21:55:20,660 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 2023-03-14 21:55:20,665 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 2023-03-14 21:55:20,670 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/d54c5885f76a4a598e92bf77ecbfa6e2 2023-03-14 21:55:21,068 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/805864f340094c76a0ec583483d89b0c 2023-03-14 21:55:21,069 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/24a7cc1266384784ac776875b0cce9ff 2023-03-14 21:55:21,085 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684 2023-03-14 21:55:21,486 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b 2023-03-14 21:55:21,486 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0 2023-03-14 21:55:21,489 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/ff2fcd642a214f4f9f08e07a2d4af155 2023-03-14 21:55:21,493 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/d54c5885f76a4a598e92bf77ecbfa6e2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2 2023-03-14 21:55:21,497 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:21,497 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/ff2fcd642a214f4f9f08e07a2d4af155 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155 2023-03-14 21:55:21,501 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:21,501 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 068a37104fe03c938dbf739e3e0c8aff in 1269ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:21,501 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:21,502 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:21,502 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:21,530 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:55:21,594 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:55:21,594 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x648a08e4 to 127.0.0.1:58882 2023-03-14 21:55:21,595 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:21,595 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3eba1225 to 127.0.0.1:58882 2023-03-14 21:55:21,595 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:21,595 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:55:21,595 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d7c4d2a to 127.0.0.1:58882 2023-03-14 21:55:21,595 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:21,595 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:55:21,595 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d1ed979 to 127.0.0.1:58882 2023-03-14 21:55:21,595 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:21,893 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/f1d2d5054f0b4eb984516db4af8dc85c 2023-03-14 21:55:21,893 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/9024cea521624ccb89d9220fbb802bab 2023-03-14 21:55:21,897 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/805864f340094c76a0ec583483d89b0c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c 2023-03-14 21:55:21,897 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/24a7cc1266384784ac776875b0cce9ff as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff 2023-03-14 21:55:21,901 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:21,901 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:21,901 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/9024cea521624ccb89d9220fbb802bab as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab 2023-03-14 21:55:21,901 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/f1d2d5054f0b4eb984516db4af8dc85c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c 2023-03-14 21:55:21,907 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:21,908 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:21,908 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f16657fdf561e3e0eb764d4e08e94891 in 1676ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:21,908 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:21,908 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:21,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:21,908 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for df07f4658e3072f4f5757b3e959f3960 in 1676ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:21,908 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:21,908 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:21,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:21,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:21,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,908 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-14 21:55:21,909 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-14 21:55:21,911 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,911 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:21,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:21,911 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:21,911 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:21,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:21,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:21,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-14 21:55:21,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d13ca76[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:21,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-14 21:55:21,913 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,914 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,914 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-14 21:55:21,914 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-14 21:55:21,914 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,914 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:21,914 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:21,918 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-14 21:55:21,918 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,918 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:21,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:21,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:21,918 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-14 21:55:21,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:21,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:21,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-14 21:55:21,920 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-14 21:55:21,920 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:21,920 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:21,920 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-14 21:55:21,920 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:21,922 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,922 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:21,922 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:21,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:21,922 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:21,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,924 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:21,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,925 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,925 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:21,925 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:21,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:21,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:21,925 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:21,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-14 21:55:21,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:21,926 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:21,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:21,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:21,926 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:22,026 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:22,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:22,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:22,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:22,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:22,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:22,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:22,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-14 21:55:22,033 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-14 21:55:22,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:22,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-14 21:55:22,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-14 21:55:22,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:22,033 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:22,034 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:22,034 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:22,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:22,035 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:22,035 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:22,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:22,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-14 21:55:22,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-14 21:55:22,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-14 21:55:22,036 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-14 21:55:22,036 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:22,036 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-14 21:55:22,036 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:22,036 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:22,036 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df07f4658e3072f4f5757b3e959f3960 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:22,040 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:22,040 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:22,040 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:22,040 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:22,040 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 068a37104fe03c938dbf739e3e0c8aff 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:22,040 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:22,040 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:22,040 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f16657fdf561e3e0eb764d4e08e94891 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:22,056 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 2023-03-14 21:55:22,064 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b 2023-03-14 21:55:22,456 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f 2023-03-14 21:55:22,472 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/1e8af449b7874e99b8f79f4cd7d62f16 2023-03-14 21:55:22,473 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/3d267d78316948f09847a76d93814116 2023-03-14 21:55:22,861 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/5190b039aa1a498aacd63c95ac39fa50 2023-03-14 21:55:22,893 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8 2023-03-14 21:55:22,895 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531 2023-03-14 21:55:23,275 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b 2023-03-14 21:55:23,298 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/6e98178865674517bf8abef7673555eb 2023-03-14 21:55:23,300 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/66f769e5545e44679e833945f42b4ac6 2023-03-14 21:55:23,301 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/1e8af449b7874e99b8f79f4cd7d62f16 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16 2023-03-14 21:55:23,304 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/3d267d78316948f09847a76d93814116 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116 2023-03-14 21:55:23,305 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:23,306 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/6e98178865674517bf8abef7673555eb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb 2023-03-14 21:55:23,308 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:23,309 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/66f769e5545e44679e833945f42b4ac6 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6 2023-03-14 21:55:23,310 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:23,311 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 068a37104fe03c938dbf739e3e0c8aff in 1271ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:23,311 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:23,311 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:23,311 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:23,312 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:23,313 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f16657fdf561e3e0eb764d4e08e94891 in 1273ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:23,313 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:23,313 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:23,313 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:23,680 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/1659b63d0821455080073630bd919a4d 2023-03-14 21:55:23,684 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/5190b039aa1a498aacd63c95ac39fa50 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50 2023-03-14 21:55:23,687 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:23,688 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/1659b63d0821455080073630bd919a4d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d 2023-03-14 21:55:23,692 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:23,693 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for df07f4658e3072f4f5757b3e959f3960 in 1657ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:23,693 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:23,693 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:23,693 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:23,693 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:23,693 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,693 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-14 21:55:23,693 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-14 21:55:23,695 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,695 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:23,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:23,695 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,695 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:23,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:23,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:23,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:23,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-14 21:55:23,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3abb5a2a[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:23,697 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-14 21:55:23,697 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,699 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,699 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-14 21:55:23,699 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-14 21:55:23,699 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:23,699 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,699 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:23,701 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,701 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-14 21:55:23,701 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:23,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:23,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:23,701 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-14 21:55:23,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:23,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:23,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:23,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:23,704 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-14 21:55:23,704 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:23,704 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-14 21:55:23,704 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-14 21:55:23,704 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:23,707 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,707 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,707 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:23,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,708 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:23,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:23,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:23,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:23,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,709 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:23,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:23,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:23,711 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:23,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:23,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-14 21:55:23,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:23,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-14 21:55:23,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:23,711 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:23,712 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:23,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:23,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:23,712 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:23,812 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:23,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:23,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:23,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678830923823 (current time:1678830923823). 2023-03-14 21:55:23,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-14 21:55:23,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] zookeeper.ReadOnlyZKClient(139): Connect 0x5f2d39e2 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:23,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9113927, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:23,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:23,837 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42970, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:23,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f2d39e2 to 127.0.0.1:58882 2023-03-14 21:55:23,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:23,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-14 21:55:23,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:23,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:23,848 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:23,848 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678831523848 2023-03-14 21:55:23,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:23,849 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678830923813 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-14 21:55:23,850 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:23,850 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-14 21:55:23,851 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-03-14 21:55:23,851 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-14 21:55:23,852 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678831523852 2023-03-14 21:55:23,854 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:23,855 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-14 21:55:23,871 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678830923813 2023-03-14 21:55:23,872 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678830923813' 2023-03-14 21:55:23,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-14 21:55:23,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678830923813' starting 'acquire' 2023-03-14 21:55:23,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678830923813', kicking off acquire phase on members. 2023-03-14 21:55:23,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:23,872 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678830923813 2023-03-14 21:55:23,874 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,874 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-14 21:55:23,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-14 21:55:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-14 21:55:23,875 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678830923813 2023-03-14 21:55:23,875 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:23,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-14 21:55:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678830923813 2023-03-14 21:55:23,876 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678830923813 from table testCompactionWithHFileLink type FLUSH 2023-03-14 21:55:23,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678830923813 2023-03-14 21:55:23,878 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678830923813' with timeout 300000ms 2023-03-14 21:55:23,878 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-14 21:55:23,878 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678830923813' starting 'acquire' stage 2023-03-14 21:55:23,878 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678830923813' locally acquired 2023-03-14 21:55:23,878 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (snaptb-1678830923813) in zk 2023-03-14 21:55:23,879 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,879 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:23,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-14 21:55:23,880 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,880 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678830923813' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:23,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:23,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:23,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:23,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:23,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'snaptb-1678830923813' on coordinator 2023-03-14 21:55:23,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678830923813' starting 'in-barrier' execution. 2023-03-14 21:55:23,881 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@474691e7[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:23,882 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:23,882 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678830923813' received 'reached' from coordinator. 2023-03-14 21:55:23,882 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:23,882 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:23,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:23,883 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:23,883 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.' region-info for snapshot=snaptb-1678830923813 2023-03-14 21:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.' region-info for snapshot=snaptb-1678830923813 2023-03-14 21:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.' region-info for snapshot=snaptb-1678830923813 2023-03-14 21:55:23,889 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-14 21:55:23,889 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-14 21:55:23,889 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116] hfiles 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16] hfiles 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50] hfiles 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11 for snapshot=snaptb-1678830923813 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9 for snapshot=snaptb-1678830923813 2023-03-14 21:55:23,893 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,062 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb] hfiles 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6] hfiles 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d] hfiles 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6 for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,065 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:24,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:24,069 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-14 21:55:24,081 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. completed. 2023-03-14 21:55:24,081 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:24,081 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-14 21:55:24,270 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:24,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:24,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:24,270 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-14 21:55:24,481 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. completed. 2023-03-14 21:55:24,481 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-14 21:55:24,483 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. completed. 2023-03-14 21:55:24,483 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:24,483 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-14 21:55:24,483 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-14 21:55:24,484 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,484 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678830923813' locally completed 2023-03-14 21:55:24,484 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678830923813' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:24,486 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678830923813' has notified controller of completion 2023-03-14 21:55:24,486 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:24,486 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,486 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678830923813' completed. 2023-03-14 21:55:24,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:24,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-14 21:55:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678830923813' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:24,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'snaptb-1678830923813', counting down latch. Waiting for 0 more 2023-03-14 21:55:24,489 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678830923813' execution completed 2023-03-14 21:55:24,489 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:24,489 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:24,489 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678830923813 2023-03-14 21:55:24,489 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678830923813including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-14 21:55:24,491 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,491 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-14 21:55:24,492 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,492 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:24,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-14 21:55:24,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:24,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:24,493 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:24,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:24,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678830923813 2023-03-14 21:55:24,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,495 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678830923813 2023-03-14 21:55:24,496 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:24,496 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678830923813 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678830923813 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678830923813/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:24,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-14 21:55:24,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-14 21:55:24,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1678830923813 because node does not exist (not an error) 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:24,496 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678830923813 2023-03-14 21:55:24,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-14 21:55:24,496 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678830923813 2023-03-14 21:55:24,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-14 21:55:24,496 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-14 21:55:24,496 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678830923813 2023-03-14 21:55:24,497 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28] mob files 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 2023-03-14 21:55:24,498 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8] mob files 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b 2023-03-14 21:55:24,499 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8 2023-03-14 21:55:24,571 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:24,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:24,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:24,572 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-14 21:55:24,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678830923813 2023-03-14 21:55:24,906 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/snaptb-1678830923813 2023-03-14 21:55:25,072 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:25,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:25,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:25,073 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-14 21:55:25,347 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/snaptb-1678830923813 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/snaptb-1678830923813 2023-03-14 21:55:26,073 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:26,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:26,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:26,074 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-14 21:55:26,167 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678830923813 of table testCompactionWithHFileLink completed 2023-03-14 21:55:26,167 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/snaptb-1678830923813 2023-03-14 21:55:26,167 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/snaptb-1678830923813 2023-03-14 21:55:26,167 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678830923813 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678830923849Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678830924904 (+1055 ms)Consolidate snapshot: snaptb-1678830923813 at 1678830924904Loading Region manifests for snaptb-1678830923813 at 1678830924904Writing data manifest for snaptb-1678830923813 at 1678830924916 (+12 ms)Verifying snapshot: snaptb-1678830923813 at 1678830925324 (+408 ms)Snapshot snaptb-1678830923813 of table testCompactionWithHFileLink completed at 1678830926167 (+843 ms) 2023-03-14 21:55:26,169 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-14 21:55:26,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.3190 sec 2023-03-14 21:55:28,075 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:28,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:28,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-14 21:55:28,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:28,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:28,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-14 21:55:28,108 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-14 21:55:28,108 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:28,108 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-14 21:55:28,108 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-14 21:55:28,108 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:28,108 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:28,109 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:28,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:28,109 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:28,110 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:28,110 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:28,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:28,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:28,111 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-14 21:55:28,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-14 21:55:28,111 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-14 21:55:28,111 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:28,111 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-14 21:55:28,112 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:28,112 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. started... 2023-03-14 21:55:28,112 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df07f4658e3072f4f5757b3e959f3960 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:28,116 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:28,116 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:28,116 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:28,116 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. started... 2023-03-14 21:55:28,116 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:28,116 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f16657fdf561e3e0eb764d4e08e94891 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:28,116 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. started... 2023-03-14 21:55:28,116 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 068a37104fe03c938dbf739e3e0c8aff 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:28,943 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/d54ec59a1231473b9ee1ac9d357ba68a 2023-03-14 21:55:28,943 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/9deda18fdc424ca3b86ace7d5cb1b801 2023-03-14 21:55:28,945 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/8f508b593099443682ed1ed6a26ed5a9 2023-03-14 21:55:28,947 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d54ec59a1231473b9ee1ac9d357ba68a 2023-03-14 21:55:28,947 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9deda18fdc424ca3b86ace7d5cb1b801 2023-03-14 21:55:28,949 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f508b593099443682ed1ed6a26ed5a9 2023-03-14 21:55:29,767 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/30d2c3e5c69e4d8f90cae26deb53e671 2023-03-14 21:55:29,767 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/fa95992412904f93be0eed33fdb368da 2023-03-14 21:55:29,770 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa95992412904f93be0eed33fdb368da 2023-03-14 21:55:29,771 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30d2c3e5c69e4d8f90cae26deb53e671 2023-03-14 21:55:29,770 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/2ebd8f3533c045fcbc99f86684e91e9c 2023-03-14 21:55:29,771 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/d54ec59a1231473b9ee1ac9d357ba68a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a 2023-03-14 21:55:29,771 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/9deda18fdc424ca3b86ace7d5cb1b801 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801 2023-03-14 21:55:29,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9deda18fdc424ca3b86ace7d5cb1b801 2023-03-14 21:55:29,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801, entries=3, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d54ec59a1231473b9ee1ac9d357ba68a 2023-03-14 21:55:29,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a, entries=3, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,775 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ebd8f3533c045fcbc99f86684e91e9c 2023-03-14 21:55:29,776 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/30d2c3e5c69e4d8f90cae26deb53e671 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671 2023-03-14 21:55:29,776 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/fa95992412904f93be0eed33fdb368da as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da 2023-03-14 21:55:29,776 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/8f508b593099443682ed1ed6a26ed5a9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9 2023-03-14 21:55:29,780 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30d2c3e5c69e4d8f90cae26deb53e671 2023-03-14 21:55:29,780 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671, entries=1, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,780 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa95992412904f93be0eed33fdb368da 2023-03-14 21:55:29,781 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da, entries=1, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,781 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f508b593099443682ed1ed6a26ed5a9 2023-03-14 21:55:29,781 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9, entries=3, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,781 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f16657fdf561e3e0eb764d4e08e94891 in 1665ms, sequenceid=24, compaction requested=true 2023-03-14 21:55:29,781 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:29,781 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:29,781 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for df07f4658e3072f4f5757b3e959f3960 in 1669ms, sequenceid=24, compaction requested=true 2023-03-14 21:55:29,782 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/2ebd8f3533c045fcbc99f86684e91e9c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c 2023-03-14 21:55:29,782 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:29,782 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:29,782 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:29,787 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ebd8f3533c045fcbc99f86684e91e9c 2023-03-14 21:55:29,787 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c, entries=1, sequenceid=24, filesize=5.0 K 2023-03-14 21:55:29,787 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 068a37104fe03c938dbf739e3e0c8aff in 1671ms, sequenceid=24, compaction requested=true 2023-03-14 21:55:29,788 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:29,788 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:29,788 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:29,788 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:29,788 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,788 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-14 21:55:29,788 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-14 21:55:29,789 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,789 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:29,790 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,790 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:29,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:29,791 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-14 21:55:29,791 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-14 21:55:29,791 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@410b6183[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:29,792 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,797 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,797 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-14 21:55:29,797 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-14 21:55:29,797 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:29,797 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,797 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:29,799 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,799 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-14 21:55:29,799 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:29,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:29,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:29,799 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-14 21:55:29,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:29,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:29,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:29,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:29,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-14 21:55:29,801 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-14 21:55:29,801 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:29,801 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:29,801 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-14 21:55:29,801 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:29,803 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:29,803 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,803 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:29,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:29,803 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,805 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-14 21:55:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:29,807 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:29,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-14 21:55:29,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:29,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-14 21:55:29,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-14 21:55:29,807 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-14 21:55:29,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-14 21:55:29,808 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:29,808 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:29,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:29,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:29,908 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:29,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:29,910 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:29,910 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): df07f4658e3072f4f5757b3e959f3960/family1 is initiating major compaction (all files) 2023-03-14 21:55:29,910 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:29,910 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:29,911 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of df07f4658e3072f4f5757b3e959f3960/family1 in testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:29,911 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp, totalSize=26.7 K 2023-03-14 21:55:29,912 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 104d41f04ac64b5d8d71fcd11b80ee11, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917428 2023-03-14 21:55:29,912 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting c106356df3774a0da464e118374ba796, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:29,912 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 805864f340094c76a0ec583483d89b0c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920216 2023-03-14 21:55:29,913 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5190b039aa1a498aacd63c95ac39fa50, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922028 2023-03-14 21:55:29,913 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d54ec59a1231473b9ee1ac9d357ba68a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:29,924 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): df07f4658e3072f4f5757b3e959f3960#family1#compaction#108 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:30,333 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del 2023-03-14 21:55:30,333 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del 2023-03-14 21:55:31,145 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0a94af7392b47e0b0e1b01c1d38426a 2023-03-14 21:55:31,146 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/c0a94af7392b47e0b0e1b01c1d38426a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c0a94af7392b47e0b0e1b01c1d38426a 2023-03-14 21:55:31,150 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0a94af7392b47e0b0e1b01c1d38426a 2023-03-14 21:55:31,151 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in df07f4658e3072f4f5757b3e959f3960/family1 of df07f4658e3072f4f5757b3e959f3960 into c0a94af7392b47e0b0e1b01c1d38426a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:31,151 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:31,151 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:31,151 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): df07f4658e3072f4f5757b3e959f3960/family2 is initiating major compaction (all files) 2023-03-14 21:55:31,152 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:31,152 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:31,152 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of df07f4658e3072f4f5757b3e959f3960/family2 in testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:31,153 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp, totalSize=25.4 K 2023-03-14 21:55:31,153 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting edd24bca9b49439b9bd49f6f9b0743af, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917428 2023-03-14 21:55:31,153 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting a881deaf5aee415990f4d1bfab3bd1c8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:31,154 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting f1d2d5054f0b4eb984516db4af8dc85c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920216 2023-03-14 21:55:31,154 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 1659b63d0821455080073630bd919a4d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922028 2023-03-14 21:55:31,154 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting fa95992412904f93be0eed33fdb368da, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:31,165 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): df07f4658e3072f4f5757b3e959f3960#family2#compaction#109 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:31,165 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a] to archive 2023-03-14 21:55:31,165 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:31,167 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/104d41f04ac64b5d8d71fcd11b80ee11 2023-03-14 21:55:31,168 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c106356df3774a0da464e118374ba796 2023-03-14 21:55:31,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/805864f340094c76a0ec583483d89b0c 2023-03-14 21:55:31,170 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/5190b039aa1a498aacd63c95ac39fa50 2023-03-14 21:55:31,171 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/d54ec59a1231473b9ee1ac9d357ba68a 2023-03-14 21:55:31,573 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del 2023-03-14 21:55:31,573 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del 2023-03-14 21:55:32,387 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99279bc9e12741bcb52739a45f0d677f 2023-03-14 21:55:32,387 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/99279bc9e12741bcb52739a45f0d677f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/99279bc9e12741bcb52739a45f0d677f 2023-03-14 21:55:32,390 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99279bc9e12741bcb52739a45f0d677f 2023-03-14 21:55:32,391 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in df07f4658e3072f4f5757b3e959f3960/family2 of df07f4658e3072f4f5757b3e959f3960 into 99279bc9e12741bcb52739a45f0d677f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:32,391 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:32,391 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:32,391 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 068a37104fe03c938dbf739e3e0c8aff/family1 is initiating major compaction (all files) 2023-03-14 21:55:32,392 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:32,392 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:32,393 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 068a37104fe03c938dbf739e3e0c8aff/family1 in testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:32,393 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp, totalSize=26.7 K 2023-03-14 21:55:32,393 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d9e4054fc8664bc08b5a5206e1eeaab9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917429 2023-03-14 21:55:32,393 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 613d0296f4cb4dfcb66cb3b44c709ce8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:32,394 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d54c5885f76a4a598e92bf77ecbfa6e2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:32,394 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 1e8af449b7874e99b8f79f4cd7d62f16, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:32,394 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 8f508b593099443682ed1ed6a26ed5a9, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:32,404 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 068a37104fe03c938dbf739e3e0c8aff#family1#compaction#110 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:32,417 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del 2023-03-14 21:55:32,417 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del 2023-03-14 21:55:32,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da] to archive 2023-03-14 21:55:32,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:32,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/edd24bca9b49439b9bd49f6f9b0743af 2023-03-14 21:55:32,469 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/a881deaf5aee415990f4d1bfab3bd1c8 2023-03-14 21:55:32,469 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/f1d2d5054f0b4eb984516db4af8dc85c 2023-03-14 21:55:32,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/1659b63d0821455080073630bd919a4d 2023-03-14 21:55:32,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/fa95992412904f93be0eed33fdb368da 2023-03-14 21:55:32,837 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36cef08b81784743ab164dad6c0a0f1c 2023-03-14 21:55:32,838 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/36cef08b81784743ab164dad6c0a0f1c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/36cef08b81784743ab164dad6c0a0f1c 2023-03-14 21:55:32,841 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36cef08b81784743ab164dad6c0a0f1c 2023-03-14 21:55:32,842 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 068a37104fe03c938dbf739e3e0c8aff/family1 of 068a37104fe03c938dbf739e3e0c8aff into 36cef08b81784743ab164dad6c0a0f1c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:32,842 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:32,842 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:32,843 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 068a37104fe03c938dbf739e3e0c8aff/family2 is initiating major compaction (all files) 2023-03-14 21:55:32,843 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:32,843 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:32,844 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 068a37104fe03c938dbf739e3e0c8aff/family2 in testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:32,844 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp, totalSize=25.4 K 2023-03-14 21:55:32,844 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 4dfcc1ed30054965a87ed75c1e9bad70, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917429 2023-03-14 21:55:32,845 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5744c93167d04cc8b606b26ae895bfc5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:32,845 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting ff2fcd642a214f4f9f08e07a2d4af155, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:32,845 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 6e98178865674517bf8abef7673555eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:32,846 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 2ebd8f3533c045fcbc99f86684e91e9c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:32,862 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 068a37104fe03c938dbf739e3e0c8aff#family2#compaction#111 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:32,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9] to archive 2023-03-14 21:55:32,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:32,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d9e4054fc8664bc08b5a5206e1eeaab9 2023-03-14 21:55:32,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/613d0296f4cb4dfcb66cb3b44c709ce8 2023-03-14 21:55:32,871 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del 2023-03-14 21:55:32,871 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del 2023-03-14 21:55:32,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d54c5885f76a4a598e92bf77ecbfa6e2 2023-03-14 21:55:32,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/1e8af449b7874e99b8f79f4cd7d62f16 2023-03-14 21:55:32,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/8f508b593099443682ed1ed6a26ed5a9 2023-03-14 21:55:33,686 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 174fe5990da144b8b9628a75fc94cfa4 2023-03-14 21:55:33,687 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/174fe5990da144b8b9628a75fc94cfa4 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/174fe5990da144b8b9628a75fc94cfa4 2023-03-14 21:55:33,691 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 174fe5990da144b8b9628a75fc94cfa4 2023-03-14 21:55:33,692 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 068a37104fe03c938dbf739e3e0c8aff/family2 of 068a37104fe03c938dbf739e3e0c8aff into 174fe5990da144b8b9628a75fc94cfa4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:33,692 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:33,692 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:33,692 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): f16657fdf561e3e0eb764d4e08e94891/family1 is initiating major compaction (all files) 2023-03-14 21:55:33,692 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:33,692 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:33,692 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of f16657fdf561e3e0eb764d4e08e94891/family1 in testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:33,692 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp, totalSize=26.7 K 2023-03-14 21:55:33,693 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5a71bf3ad2744ecebe89e0ad69ab75ed, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917430 2023-03-14 21:55:33,693 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting e92e43a4246c4a4e8dae31c701c4de2f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:33,693 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 24a7cc1266384784ac776875b0cce9ff, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:33,694 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 3d267d78316948f09847a76d93814116, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:33,694 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9deda18fdc424ca3b86ace7d5cb1b801, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:33,704 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): f16657fdf561e3e0eb764d4e08e94891#family1#compaction#112 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:33,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c] to archive 2023-03-14 21:55:33,766 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:33,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/4dfcc1ed30054965a87ed75c1e9bad70 2023-03-14 21:55:33,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/5744c93167d04cc8b606b26ae895bfc5 2023-03-14 21:55:33,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/ff2fcd642a214f4f9f08e07a2d4af155 2023-03-14 21:55:33,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/6e98178865674517bf8abef7673555eb 2023-03-14 21:55:33,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/2ebd8f3533c045fcbc99f86684e91e9c 2023-03-14 21:55:34,114 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del 2023-03-14 21:55:34,114 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del 2023-03-14 21:55:34,927 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126dbf2b147d4eb8bc14f35c51d35904 2023-03-14 21:55:34,928 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/126dbf2b147d4eb8bc14f35c51d35904 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/126dbf2b147d4eb8bc14f35c51d35904 2023-03-14 21:55:34,932 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 126dbf2b147d4eb8bc14f35c51d35904 2023-03-14 21:55:34,933 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f16657fdf561e3e0eb764d4e08e94891/family1 of f16657fdf561e3e0eb764d4e08e94891 into 126dbf2b147d4eb8bc14f35c51d35904(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:34,933 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:34,933 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:34,933 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): f16657fdf561e3e0eb764d4e08e94891/family2 is initiating major compaction (all files) 2023-03-14 21:55:34,933 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:34,933 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:34,934 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of f16657fdf561e3e0eb764d4e08e94891/family2 in testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:34,934 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp, totalSize=25.4 K 2023-03-14 21:55:34,934 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting e8d4a324c2e54e328bd67e2b2377a1ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917430 2023-03-14 21:55:34,935 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 6de3f715832e4389b1a7cc6d9dc660c4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:34,935 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9024cea521624ccb89d9220fbb802bab, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:34,935 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 66f769e5545e44679e833945f42b4ac6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:34,935 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 30d2c3e5c69e4d8f90cae26deb53e671, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-14 21:55:34,945 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): f16657fdf561e3e0eb764d4e08e94891#family2#compaction#113 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:34,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801] to archive 2023-03-14 21:55:34,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:34,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/5a71bf3ad2744ecebe89e0ad69ab75ed 2023-03-14 21:55:34,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/e92e43a4246c4a4e8dae31c701c4de2f 2023-03-14 21:55:34,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/24a7cc1266384784ac776875b0cce9ff 2023-03-14 21:55:34,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/3d267d78316948f09847a76d93814116 2023-03-14 21:55:34,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/9deda18fdc424ca3b86ace7d5cb1b801 2023-03-14 21:55:35,358 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del 2023-03-14 21:55:35,358 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del 2023-03-14 21:55:36,172 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7ade920f5074ed5a4f6a66278de304c 2023-03-14 21:55:36,173 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/e7ade920f5074ed5a4f6a66278de304c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e7ade920f5074ed5a4f6a66278de304c 2023-03-14 21:55:36,177 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7ade920f5074ed5a4f6a66278de304c 2023-03-14 21:55:36,178 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f16657fdf561e3e0eb764d4e08e94891/family2 of f16657fdf561e3e0eb764d4e08e94891 into e7ade920f5074ed5a4f6a66278de304c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:36,178 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:36,183 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:36,186 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:36,189 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:36,190 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:36,199 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:55:36,232 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-14 21:55:36,235 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del 2023-03-14 21:55:36,238 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del 2023-03-14 21:55:36,241 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del 2023-03-14 21:55:36,241 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:36,241 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(139): Connect 0x2d67da39 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:36,250 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28a98f9d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:36,250 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:36,251 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:36,251 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:36,267 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671] to archive 2023-03-14 21:55:36,268 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:36,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e8d4a324c2e54e328bd67e2b2377a1ea 2023-03-14 21:55:36,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/6de3f715832e4389b1a7cc6d9dc660c4 2023-03-14 21:55:36,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/9024cea521624ccb89d9220fbb802bab 2023-03-14 21:55:36,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/66f769e5545e44679e833945f42b4ac6 2023-03-14 21:55:36,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/30d2c3e5c69e4d8f90cae26deb53e671 2023-03-14 21:55:36,737 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c 2023-03-14 21:55:36,738 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:36,752 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:36,756 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43740, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:36,762 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:36,764 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41894, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:36,772 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/9480ffda64f44c059492d359a403cb9f first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-14 21:55:36,773 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/9480ffda64f44c059492d359a403cb9f}] 2023-03-14 21:55:36,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/9480ffda64f44c059492d359a403cb9f for inclusion in 068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:36,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-14 21:55:36,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-14 21:55:36,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:36,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/9480ffda64f44c059492d359a403cb9f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__vt36cplnt8n6o0s39kvbn2cugk1ip3929rfnfkqlttg4u1q9o8cv12u7ues79ocb/family1/9480ffda64f44c059492d359a403cb9f 2023-03-14 21:55:36,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__vt36cplnt8n6o0s39kvbn2cugk1ip3929rfnfkqlttg4u1q9o8cv12u7ues79ocb/family1/9480ffda64f44c059492d359a403cb9f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ 2023-03-14 21:55:36,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__vt36cplnt8n6o0s39kvbn2cugk1ip3929rfnfkqlttg4u1q9o8cv12u7ues79ocb/family1/9480ffda64f44c059492d359a403cb9f into 068a37104fe03c938dbf739e3e0c8aff/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ - updating store file list. 2023-03-14 21:55:36,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ into 068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:36,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__vt36cplnt8n6o0s39kvbn2cugk1ip3929rfnfkqlttg4u1q9o8cv12u7ues79ocb/family1/9480ffda64f44c059492d359a403cb9f into 068a37104fe03c938dbf739e3e0c8aff/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_) 2023-03-14 21:55:36,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__vt36cplnt8n6o0s39kvbn2cugk1ip3929rfnfkqlttg4u1q9o8cv12u7ues79ocb/family1/9480ffda64f44c059492d359a403cb9f 2023-03-14 21:55:36,787 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:36,787 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:36,789 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a 2023-03-14 21:55:36,790 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 2023-03-14 21:55:36,791 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 2023-03-14 21:55:36,792 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 2023-03-14 21:55:36,792 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:36,792 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314' 2023-03-14 21:55:36,792 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:37,137 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941 2023-03-14 21:55:37,137 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396 2023-03-14 21:55:37,138 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:37,138 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:37,168 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/84dc34fff0b84de395f33bef2cc2b479 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-14 21:55:37,170 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/84dc34fff0b84de395f33bef2cc2b479}] 2023-03-14 21:55:37,172 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/156445d99e014bf09bb980b8c0f295c1 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-14 21:55:37,172 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/156445d99e014bf09bb980b8c0f295c1}] 2023-03-14 21:55:37,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/84dc34fff0b84de395f33bef2cc2b479 for inclusion in f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:37,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/156445d99e014bf09bb980b8c0f295c1 for inclusion in df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:37,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-14 21:55:37,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-14 21:55:37,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:37,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/84dc34fff0b84de395f33bef2cc2b479 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__sje10qv0rh0lm0hb0v7dfplugl3kf8v4jv13719m0effqeppg4upn0plk9ro8672/family1/84dc34fff0b84de395f33bef2cc2b479 2023-03-14 21:55:37,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-14 21:55:37,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last=1 2023-03-14 21:55:37,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:37,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/156445d99e014bf09bb980b8c0f295c1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__c12hi036d1umaubisbdg8m56tie05ku85frqkb4g183al3oi0chadg4dku60r03i/family1/156445d99e014bf09bb980b8c0f295c1 2023-03-14 21:55:37,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__sje10qv0rh0lm0hb0v7dfplugl3kf8v4jv13719m0effqeppg4upn0plk9ro8672/family1/84dc34fff0b84de395f33bef2cc2b479 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ 2023-03-14 21:55:37,180 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__sje10qv0rh0lm0hb0v7dfplugl3kf8v4jv13719m0effqeppg4upn0plk9ro8672/family1/84dc34fff0b84de395f33bef2cc2b479 into f16657fdf561e3e0eb764d4e08e94891/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ - updating store file list. 2023-03-14 21:55:37,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__c12hi036d1umaubisbdg8m56tie05ku85frqkb4g183al3oi0chadg4dku60r03i/family1/156445d99e014bf09bb980b8c0f295c1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ 2023-03-14 21:55:37,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__c12hi036d1umaubisbdg8m56tie05ku85frqkb4g183al3oi0chadg4dku60r03i/family1/156445d99e014bf09bb980b8c0f295c1 into df07f4658e3072f4f5757b3e959f3960/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ - updating store file list. 2023-03-14 21:55:37,183 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ into f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:37,183 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__sje10qv0rh0lm0hb0v7dfplugl3kf8v4jv13719m0effqeppg4upn0plk9ro8672/family1/84dc34fff0b84de395f33bef2cc2b479 into f16657fdf561e3e0eb764d4e08e94891/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_) 2023-03-14 21:55:37,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__sje10qv0rh0lm0hb0v7dfplugl3kf8v4jv13719m0effqeppg4upn0plk9ro8672/family1/84dc34fff0b84de395f33bef2cc2b479 2023-03-14 21:55:37,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ into df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:37,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__c12hi036d1umaubisbdg8m56tie05ku85frqkb4g183al3oi0chadg4dku60r03i/family1/156445d99e014bf09bb980b8c0f295c1 into df07f4658e3072f4f5757b3e959f3960/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_) 2023-03-14 21:55:37,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__c12hi036d1umaubisbdg8m56tie05ku85frqkb4g183al3oi0chadg4dku60r03i/family1/156445d99e014bf09bb980b8c0f295c1 2023-03-14 21:55:37,185 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:37,186 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:37,187 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:37,187 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:37,188 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 2023-03-14 21:55:37,188 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 2023-03-14 21:55:37,189 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 2023-03-14 21:55:37,190 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f 2023-03-14 21:55:37,190 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b 2023-03-14 21:55:37,191 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 2023-03-14 21:55:37,191 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 2023-03-14 21:55:37,191 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:37,191 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314' 2023-03-14 21:55:37,192 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 2023-03-14 21:55:37,192 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:37,192 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:37,192 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314' 2023-03-14 21:55:37,192 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:37,192 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:37,192 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:37,192 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del] 2023-03-14 21:55:37,193 DEBUG [Listener at localhost/45975] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:37,194 DEBUG [Listener at localhost/45975] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303145166bf811fd548dfbde54464eca8fcda_del 2023-03-14 21:55:37,194 INFO [Listener at localhost/45975] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-14 21:55:37,194 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del] 2023-03-14 21:55:37,194 DEBUG [Listener at localhost/45975] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:37,196 DEBUG [Listener at localhost/45975] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303145377f4de6eb64ad28e5fe7b5c6439bc6_del 2023-03-14 21:55:37,196 INFO [Listener at localhost/45975] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-14 21:55:37,196 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del] 2023-03-14 21:55:37,196 DEBUG [Listener at localhost/45975] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:37,197 DEBUG [Listener at localhost/45975] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303146c2ac7249c334f83a7c10b75bcf047c6_del 2023-03-14 21:55:37,197 INFO [Listener at localhost/45975] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-14 21:55:37,199 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:37,202 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:37,205 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:37,206 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:37,212 INFO [Listener at localhost/45975] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-14 21:55:37,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-14 21:55:37,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-14 21:55:37,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-14 21:55:37,215 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830937215"}]},"ts":"1678830937215"} 2023-03-14 21:55:37,216 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-14 21:55:37,218 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-14 21:55:37,219 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, UNASSIGN}] 2023-03-14 21:55:37,220 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, UNASSIGN 2023-03-14 21:55:37,220 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, UNASSIGN 2023-03-14 21:55:37,220 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, UNASSIGN 2023-03-14 21:55:37,221 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:37,221 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:37,221 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:37,221 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830937221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830937221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830937221"}]},"ts":"1678830937221"} 2023-03-14 21:55:37,221 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830937221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830937221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830937221"}]},"ts":"1678830937221"} 2023-03-14 21:55:37,221 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830937221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830937221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830937221"}]},"ts":"1678830937221"} 2023-03-14 21:55:37,222 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:37,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:37,223 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:37,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-14 21:55:37,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:37,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing df07f4658e3072f4f5757b3e959f3960, disabling compactions & flushes 2023-03-14 21:55:37,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:37,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:37,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. after waiting 0 ms 2023-03-14 21:55:37,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:37,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-14 21:55:37,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:37,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:37,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:37,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:37,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f16657fdf561e3e0eb764d4e08e94891, disabling compactions & flushes 2023-03-14 21:55:37,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:37,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:37,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. after waiting 0 ms 2023-03-14 21:55:37,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:37,381 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=CLOSED 2023-03-14 21:55:37,381 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830937381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830937381"}]},"ts":"1678830937381"} 2023-03-14 21:55:37,384 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-03-14 21:55:37,384 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197 in 161 msec 2023-03-14 21:55:37,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-14 21:55:37,385 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:37,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:37,386 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, UNASSIGN in 165 msec 2023-03-14 21:55:37,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:37,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:37,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 068a37104fe03c938dbf739e3e0c8aff, disabling compactions & flushes 2023-03-14 21:55:37,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:37,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:37,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. after waiting 0 ms 2023-03-14 21:55:37,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:37,387 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=CLOSED 2023-03-14 21:55:37,387 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830937387"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830937387"}]},"ts":"1678830937387"} 2023-03-14 21:55:37,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-14 21:55:37,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197 in 165 msec 2023-03-14 21:55:37,390 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, UNASSIGN in 170 msec 2023-03-14 21:55:37,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-14 21:55:37,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:37,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:37,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:37,393 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=CLOSED 2023-03-14 21:55:37,393 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830937393"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830937393"}]},"ts":"1678830937393"} 2023-03-14 21:55:37,395 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-14 21:55:37,395 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197 in 171 msec 2023-03-14 21:55:37,396 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-14 21:55:37,396 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, UNASSIGN in 176 msec 2023-03-14 21:55:37,396 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830937396"}]},"ts":"1678830937396"} 2023-03-14 21:55:37,397 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-14 21:55:37,398 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-14 21:55:37,400 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 187 msec 2023-03-14 21:55:37,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-14 21:55:37,516 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-14 21:55:37,526 INFO [Listener at localhost/45975] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:37,527 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:37,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678830937527 (current time:1678830937527). 2023-03-14 21:55:37,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-14 21:55:37,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] zookeeper.ReadOnlyZKClient(139): Connect 0x37f9c2e8 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:37,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2075077b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:37,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:37,535 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43754, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:37,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37f9c2e8 to 127.0.0.1:58882 2023-03-14 21:55:37,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:37,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-14 21:55:37,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:37,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:37,539 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:37,540 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678831537540 2023-03-14 21:55:37,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:37,540 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678830923813-1678830937526 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-14 21:55:37,540 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:37,540 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-14 21:55:37,640 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:37,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:37,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:37,642 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-14 21:55:37,842 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:37,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:37,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:37,843 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-14 21:55:37,946 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-14 21:55:37,948 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c0a94af7392b47e0b0e1b01c1d38426a] hfiles 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941] hfiles 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/126dbf2b147d4eb8bc14f35c51d35904, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_] hfiles 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/36cef08b81784743ab164dad6c0a0f1c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_] hfiles 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/36cef08b81784743ab164dad6c0a0f1c 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/126dbf2b147d4eb8bc14f35c51d35904 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ 2023-03-14 21:55:37,951 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c0a94af7392b47e0b0e1b01c1d38426a 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e7ade920f5074ed5a4f6a66278de304c] hfiles 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e7ade920f5074ed5a4f6a66278de304c 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/99279bc9e12741bcb52739a45f0d677f] hfiles 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/99279bc9e12741bcb52739a45f0d677f 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/174fe5990da144b8b9628a75fc94cfa4] hfiles 2023-03-14 21:55:37,952 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/174fe5990da144b8b9628a75fc94cfa4 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8] hfiles 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303143a8374c011fb48ed9ee050b648119a53 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023031444e3f9c04a72497c962013b9aed711e8 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314bcd7670380c74417bb2f206112d56a85 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314d1311d6bc2a04bff80af630a288f8684 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303143c634c99823448f4998e29293b754b8e 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453ac49e32bb14884a0053209f857eea0 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031498a9d4a906184b9e91382753c23da531 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230314ea0107fb0b5e4651b0aad7798f1b847c 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314001cf6c83b4c4861ac1b47372932870b 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303143b64cfe656154c3b9dab650f24020d45 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ed446e6247764ad7ae651adc2d59679b 2023-03-14 21:55:37,958 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230314ee9ee9adbc90473e8398eb3d84c9a1e8 2023-03-14 21:55:38,143 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:38,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:38,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:38,145 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-14 21:55:38,361 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-14 21:55:38,362 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:38,362 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:38,645 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:38,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:38,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-14 21:55:38,646 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-14 21:55:38,786 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678830923813-1678830937526 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:39,601 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678830923813-1678830937526 of table testCompactionWithHFileLink completed 2023-03-14 21:55:39,601 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:39,602 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:39,602 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678830923813-1678830937526 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678830937540Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678830937946 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678830923813-1678830937526 at 1678830938362 (+416 ms)Loading Region manifests for hbase-failsafe-snaptb-1678830923813-1678830937526 at 1678830938362Writing data manifest for hbase-failsafe-snaptb-1678830923813-1678830937526 at 1678830938364 (+2 ms)Verifying snapshot: hbase-failsafe-snaptb-1678830923813-1678830937526 at 1678830938770 (+406 ms)Snapshot hbase-failsafe-snaptb-1678830923813-1678830937526 of table testCompactionWithHFileLink completed at 1678830939601 (+831 ms) 2023-03-14 21:55:39,603 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-14 21:55:39,604 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0650 sec 2023-03-14 21:55:39,646 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-14 21:55:39,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-14 21:55:39,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678830923813-1678830937526 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-14 21:55:39,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678830923813" table: "testCompactionWithHFileLink" creation_time: 1678830923823 type: FLUSH version: 2 ttl: 0 ) 2023-03-14 21:55:39,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678830923813 as table=testCompactionWithHFileLink 2023-03-14 21:55:39,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-14 21:55:39,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-14 21:55:39,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-14 21:55:40,095 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-14 21:55:40,096 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-14 21:55:40,096 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-14 21:55:40,097 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-14 21:55:40,101 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678830923813" table: "testCompactionWithHFileLink" creation_time: 1678830923823 type: FLUSH version: 2 ttl: 0 2023-03-14 21:55:40,101 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink 2023-03-14 21:55:40,106 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-14 21:55:40,109 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023031460eb07189ff84968bc944cb1e4264941 2023-03-14 21:55:40,110 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031460148c6c2403463eb7fbb72a0bd1274c 2023-03-14 21:55:40,111 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303147c2a906959944512b9629073cb84d396 2023-03-14 21:55:40,111 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,114 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,117 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,120 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,122 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,125 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,128 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,131 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,134 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,137 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,140 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,142 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 of snapshot=snaptb-1678830923813 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-14 21:55:40,147 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230314a0507fe5d54d439d9aecde1f0f47276d_del 2023-03-14 21:55:40,148 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023031453dfbbe838e94a3286b5fb46b6e21b85_del 2023-03-14 21:55:40,149 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303144ae2f51f3a334acda40b2fcbc0f95049_del 2023-03-14 21:55:40,149 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,149 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,149 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,153 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/36cef08b81784743ab164dad6c0a0f1c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/36cef08b81784743ab164dad6c0a0f1c 2023-03-14 21:55:40,153 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/2cda22c418614a39a81d5ed5b69d6d9e_SeqId_29_ 2023-03-14 21:55:40,153 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/126dbf2b147d4eb8bc14f35c51d35904 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/126dbf2b147d4eb8bc14f35c51d35904 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/6c0f05fee08c40c58385e0d8fb4e2e11_SeqId_29_ 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d9e4054fc8664bc08b5a5206e1eeaab9 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c0a94af7392b47e0b0e1b01c1d38426a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/c0a94af7392b47e0b0e1b01c1d38426a 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 104d41f04ac64b5d8d71fcd11b80ee11 of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/bb0b6de432b9454d820c65e0ddb5c9c9_SeqId_29_ 2023-03-14 21:55:40,154 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5a71bf3ad2744ecebe89e0ad69ab75ed of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,158 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c106356df3774a0da464e118374ba796 of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,158 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 613d0296f4cb4dfcb66cb3b44c709ce8 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,159 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e92e43a4246c4a4e8dae31c701c4de2f of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,162 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 805864f340094c76a0ec583483d89b0c of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,162 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 24a7cc1266384784ac776875b0cce9ff of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,162 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d54c5885f76a4a598e92bf77ecbfa6e2 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,165 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5190b039aa1a498aacd63c95ac39fa50 of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,165 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3d267d78316948f09847a76d93814116 of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,165 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1e8af449b7874e99b8f79f4cd7d62f16 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,174 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/99279bc9e12741bcb52739a45f0d677f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/99279bc9e12741bcb52739a45f0d677f 2023-03-14 21:55:40,174 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink edd24bca9b49439b9bd49f6f9b0743af of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,175 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/174fe5990da144b8b9628a75fc94cfa4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/174fe5990da144b8b9628a75fc94cfa4 2023-03-14 21:55:40,175 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e7ade920f5074ed5a4f6a66278de304c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/e7ade920f5074ed5a4f6a66278de304c 2023-03-14 21:55:40,175 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4dfcc1ed30054965a87ed75c1e9bad70 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,175 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e8d4a324c2e54e328bd67e2b2377a1ea of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,184 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a881deaf5aee415990f4d1bfab3bd1c8 of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,184 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5744c93167d04cc8b606b26ae895bfc5 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,185 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6de3f715832e4389b1a7cc6d9dc660c4 of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,188 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f1d2d5054f0b4eb984516db4af8dc85c of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,188 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ff2fcd642a214f4f9f08e07a2d4af155 of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,189 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9024cea521624ccb89d9220fbb802bab of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,191 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1659b63d0821455080073630bd919a4d of snapshot=snaptb-1678830923813 to region=df07f4658e3072f4f5757b3e959f3960 table=testCompactionWithHFileLink 2023-03-14 21:55:40,191 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6e98178865674517bf8abef7673555eb of snapshot=snaptb-1678830923813 to region=068a37104fe03c938dbf739e3e0c8aff table=testCompactionWithHFileLink 2023-03-14 21:55:40,192 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 66f769e5545e44679e833945f42b4ac6 of snapshot=snaptb-1678830923813 to region=f16657fdf561e3e0eb764d4e08e94891 table=testCompactionWithHFileLink 2023-03-14 21:55:40,196 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678830923813" table: "testCompactionWithHFileLink" creation_time: 1678830923823 type: FLUSH version: 2 ttl: 0 2023-03-14 21:55:40,197 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678830940197"}]},"ts":"9223372036854775807"} 2023-03-14 21:55:40,197 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678830940197"}]},"ts":"9223372036854775807"} 2023-03-14 21:55:40,197 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678830940197"}]},"ts":"9223372036854775807"} 2023-03-14 21:55:40,199 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-14 21:55:40,199 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 068a37104fe03c938dbf739e3e0c8aff, NAME => 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => df07f4658e3072f4f5757b3e959f3960, NAME => 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => f16657fdf561e3e0eb764d4e08e94891, NAME => 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.', STARTKEY => '2', ENDKEY => ''}] 2023-03-14 21:55:40,199 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830940198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830940198"}]},"ts":"1678830940198"} 2023-03-14 21:55:40,199 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830940198"}]},"ts":"1678830940198"} 2023-03-14 21:55:40,199 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830940198"}]},"ts":"1678830940198"} 2023-03-14 21:55:40,201 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-14 21:55:40,201 INFO [PEWorker-5] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-14 21:55:40,201 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 068a37104fe03c938dbf739e3e0c8aff, NAME => 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => df07f4658e3072f4f5757b3e959f3960, NAME => 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => f16657fdf561e3e0eb764d4e08e94891, NAME => 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.', STARTKEY => '2', ENDKEY => ''}] 2023-03-14 21:55:40,201 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678830923813 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-14 21:55:40,203 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678830923813" table: "testCompactionWithHFileLink" creation_time: 1678830923823 type: FLUSH version: 2 ttl: 0 ) in 526 msec 2023-03-14 21:55:40,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-14 21:55:40,292 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-14 21:55:40,292 INFO [Listener at localhost/45975] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:40,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678830923813-1678830937526" 2023-03-14 21:55:40,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678830923813-1678830937526 2023-03-14 21:55:40,300 INFO [Listener at localhost/45975] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-14 21:55:40,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-14 21:55:40,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-14 21:55:40,313 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-14 21:55:40,313 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830940313"}]},"ts":"1678830940313"} 2023-03-14 21:55:40,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-14 21:55:40,314 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-14 21:55:40,316 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-14 21:55:40,316 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN}] 2023-03-14 21:55:40,317 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN 2023-03-14 21:55:40,317 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN 2023-03-14 21:55:40,318 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN 2023-03-14 21:55:40,318 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-14 21:55:40,318 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-14 21:55:40,318 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-14 21:55:40,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-14 21:55:40,470 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,470 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,470 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,470 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830940469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830940469"}]},"ts":"1678830940469"} 2023-03-14 21:55:40,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830940469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830940469"}]},"ts":"1678830940469"} 2023-03-14 21:55:40,470 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830940469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830940469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830940469"}]},"ts":"1678830940469"} 2023-03-14 21:55:40,471 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:40,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:40,473 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:40,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-14 21:55:40,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:40,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 068a37104fe03c938dbf739e3e0c8aff, NAME => 'testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.', STARTKEY => '1', ENDKEY => '2'} 2023-03-14 21:55:40,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:40,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,627 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,628 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:40,628 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:40,628 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a37104fe03c938dbf739e3e0c8aff columnFamilyName family1 2023-03-14 21:55:40,644 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16 2023-03-14 21:55:40,648 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8 2023-03-14 21:55:40,653 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2 2023-03-14 21:55:40,657 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9 2023-03-14 21:55:40,657 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(310): Store=068a37104fe03c938dbf739e3e0c8aff/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,657 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,658 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2 2023-03-14 21:55:40,658 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2 2023-03-14 21:55:40,658 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a37104fe03c938dbf739e3e0c8aff columnFamilyName family2 2023-03-14 21:55:40,664 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70 2023-03-14 21:55:40,670 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5 2023-03-14 21:55:40,675 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb 2023-03-14 21:55:40,683 DEBUG [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155 2023-03-14 21:55:40,684 INFO [StoreOpener-068a37104fe03c938dbf739e3e0c8aff-1] regionserver.HStore(310): Store=068a37104fe03c938dbf739e3e0c8aff/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:40,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:55:40,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 068a37104fe03c938dbf739e3e0c8aff; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75459507, jitterRate=0.12443427741527557}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:40,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:40,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., pid=103, masterSystemTime=1678830940623 2023-03-14 21:55:40,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:40,690 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-14 21:55:40,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:40,690 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:40,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:40,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:40,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=068a37104fe03c938dbf739e3e0c8aff, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => df07f4658e3072f4f5757b3e959f3960, NAME => 'testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.', STARTKEY => '', ENDKEY => '1'} 2023-03-14 21:55:40,692 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830940692"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830940692"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830940692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830940692"}]},"ts":"1678830940692"} 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,694 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-14 21:55:40,694 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1912): 068a37104fe03c938dbf739e3e0c8aff/family1 is initiating minor compaction (all files) 2023-03-14 21:55:40,694 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,694 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 068a37104fe03c938dbf739e3e0c8aff/family1 in testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:40,694 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp, totalSize=21.6 K 2023-03-14 21:55:40,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-14 21:55:40,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 068a37104fe03c938dbf739e3e0c8aff, server=jenkins-hbase4.apache.org,35143,1678830821197 in 220 msec 2023-03-14 21:55:40,695 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917429 2023-03-14 21:55:40,695 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-14 21:55:40,695 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1912): 068a37104fe03c938dbf739e3e0c8aff/family2 is initiating minor compaction (all files) 2023-03-14 21:55:40,696 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 068a37104fe03c938dbf739e3e0c8aff/family2 in testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:55:40,696 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp, totalSize=20.4 K 2023-03-14 21:55:40,696 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:40,696 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:40,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=068a37104fe03c938dbf739e3e0c8aff, ASSIGN in 379 msec 2023-03-14 21:55:40,696 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:40,696 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df07f4658e3072f4f5757b3e959f3960 columnFamilyName family1 2023-03-14 21:55:40,696 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917429 2023-03-14 21:55:40,697 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:40,697 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:40,698 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:40,698 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:40,700 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:40,712 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11 2023-03-14 21:55:40,729 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50 2023-03-14 21:55:40,738 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 068a37104fe03c938dbf739e3e0c8aff#family1#compaction#114 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:40,739 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c 2023-03-14 21:55:40,746 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] throttle.PressureAwareThroughputController(145): 068a37104fe03c938dbf739e3e0c8aff#family2#compaction#115 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:40,750 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796 2023-03-14 21:55:40,752 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(310): Store=df07f4658e3072f4f5757b3e959f3960/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,752 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,753 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2 2023-03-14 21:55:40,753 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2 2023-03-14 21:55:40,754 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df07f4658e3072f4f5757b3e959f3960 columnFamilyName family2 2023-03-14 21:55:40,760 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d 2023-03-14 21:55:40,763 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8 2023-03-14 21:55:40,767 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af 2023-03-14 21:55:40,771 DEBUG [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c 2023-03-14 21:55:40,771 INFO [StoreOpener-df07f4658e3072f4f5757b3e959f3960-1] regionserver.HStore(310): Store=df07f4658e3072f4f5757b3e959f3960/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:40,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:55:40,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened df07f4658e3072f4f5757b3e959f3960; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67397204, jitterRate=0.004296600818634033}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:40,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:40,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., pid=102, masterSystemTime=1678830940623 2023-03-14 21:55:40,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-14 21:55:40,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-14 21:55:40,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:40,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:40,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:40,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f16657fdf561e3e0eb764d4e08e94891, NAME => 'testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.', STARTKEY => '2', ENDKEY => ''} 2023-03-14 21:55:40,778 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=df07f4658e3072f4f5757b3e959f3960, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,778 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940777"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830940777"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830940777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830940777"}]},"ts":"1678830940777"} 2023-03-14 21:55:40,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:40,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,779 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,780 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:40,780 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:40,780 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-14 21:55:40,780 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f16657fdf561e3e0eb764d4e08e94891 columnFamilyName family1 2023-03-14 21:55:40,780 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure df07f4658e3072f4f5757b3e959f3960, server=jenkins-hbase4.apache.org,35143,1678830821197 in 307 msec 2023-03-14 21:55:40,781 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=df07f4658e3072f4f5757b3e959f3960, ASSIGN in 464 msec 2023-03-14 21:55:40,786 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff 2023-03-14 21:55:40,791 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116 2023-03-14 21:55:40,795 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed 2023-03-14 21:55:40,800 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f 2023-03-14 21:55:40,800 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(310): Store=f16657fdf561e3e0eb764d4e08e94891/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,801 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,801 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2 2023-03-14 21:55:40,801 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2 2023-03-14 21:55:40,802 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f16657fdf561e3e0eb764d4e08e94891 columnFamilyName family2 2023-03-14 21:55:40,808 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6 2023-03-14 21:55:40,814 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4 2023-03-14 21:55:40,820 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab 2023-03-14 21:55:40,825 DEBUG [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(539): loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea 2023-03-14 21:55:40,825 INFO [StoreOpener-f16657fdf561e3e0eb764d4e08e94891-1] regionserver.HStore(310): Store=f16657fdf561e3e0eb764d4e08e94891/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:40,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:40,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:55:40,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f16657fdf561e3e0eb764d4e08e94891; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72546863, jitterRate=0.08103249967098236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:40,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:40,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., pid=101, masterSystemTime=1678830940623 2023-03-14 21:55:40,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-14 21:55:40,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-14 21:55:40,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:40,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:40,832 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=f16657fdf561e3e0eb764d4e08e94891, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:40,833 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678830940832"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830940832"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830940832"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830940832"}]},"ts":"1678830940832"} 2023-03-14 21:55:40,835 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-14 21:55:40,835 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure f16657fdf561e3e0eb764d4e08e94891, server=jenkins-hbase4.apache.org,35143,1678830821197 in 363 msec 2023-03-14 21:55:40,838 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-03-14 21:55:40,838 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f16657fdf561e3e0eb764d4e08e94891, ASSIGN in 519 msec 2023-03-14 21:55:40,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830940838"}]},"ts":"1678830940838"} 2023-03-14 21:55:40,840 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-14 21:55:40,841 INFO [PEWorker-1] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-14 21:55:40,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 536 msec 2023-03-14 21:55:40,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-14 21:55:40,916 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-14 21:55:40,928 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:40,938 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:40,945 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:40,946 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:40,952 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:55:40,964 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-14 21:55:40,965 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:40,965 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(139): Connect 0x7e943a5b to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:40,969 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62df6183, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:40,969 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:40,969 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:40,969 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:41,047 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314d55bd98a48d341d2949bb4bc3b4ce1c3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230314d55bd98a48d341d2949bb4bc3b4ce1c3 2023-03-14 21:55:41,048 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,060 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:41,064 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43756, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:41,070 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:41,073 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41896, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:41,081 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/65659e5ccf8a4b669dda2326c3e3d967 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-14 21:55:41,082 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=32 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/65659e5ccf8a4b669dda2326c3e3d967}] 2023-03-14 21:55:41,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/65659e5ccf8a4b669dda2326c3e3d967 for inclusion in df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:41,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-14 21:55:41,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last=1 2023-03-14 21:55:41,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:41,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314/family1/65659e5ccf8a4b669dda2326c3e3d967 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__db111igco3l1bhqdeqo4dp0hgq6ikl68tdp1gjf1cvqrj68fqmgmgb2t8k9bq6uf/family1/65659e5ccf8a4b669dda2326c3e3d967 2023-03-14 21:55:41,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__db111igco3l1bhqdeqo4dp0hgq6ikl68tdp1gjf1cvqrj68fqmgmgb2t8k9bq6uf/family1/65659e5ccf8a4b669dda2326c3e3d967 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_ 2023-03-14 21:55:41,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__db111igco3l1bhqdeqo4dp0hgq6ikl68tdp1gjf1cvqrj68fqmgmgb2t8k9bq6uf/family1/65659e5ccf8a4b669dda2326c3e3d967 into df07f4658e3072f4f5757b3e959f3960/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_ - updating store file list. 2023-03-14 21:55:41,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_ into df07f4658e3072f4f5757b3e959f3960/family1 2023-03-14 21:55:41,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__db111igco3l1bhqdeqo4dp0hgq6ikl68tdp1gjf1cvqrj68fqmgmgb2t8k9bq6uf/family1/65659e5ccf8a4b669dda2326c3e3d967 into df07f4658e3072f4f5757b3e959f3960/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_) 2023-03-14 21:55:41,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__db111igco3l1bhqdeqo4dp0hgq6ikl68tdp1gjf1cvqrj68fqmgmgb2t8k9bq6uf/family1/65659e5ccf8a4b669dda2326c3e3d967 2023-03-14 21:55:41,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:41,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-14 21:55:41,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(1912): df07f4658e3072f4f5757b3e959f3960/family1 is initiating minor compaction (all files) 2023-03-14 21:55:41,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5993a532; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-14 21:55:41,099 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,100 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:41,101 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314030c0ee58d16424dbe3d50794c908607 2023-03-14 21:55:41,102 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023031485df72141051461da9d3495ced9ae06f 2023-03-14 21:55:41,103 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314ae76584737094322bc97e33c43428bc9 2023-03-14 21:55:41,103 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230314e480c293770f4fc39d954f1dd0eb7f28 2023-03-14 21:55:41,104 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:41,104 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230314' 2023-03-14 21:55:41,104 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:41,562 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family1/3825b672290a4ca48a1fbf8fbc84512e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/3825b672290a4ca48a1fbf8fbc84512e 2023-03-14 21:55:41,567 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 068a37104fe03c938dbf739e3e0c8aff/family1 of 068a37104fe03c938dbf739e3e0c8aff into 3825b672290a4ca48a1fbf8fbc84512e(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:41,567 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:41,567 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., storeName=068a37104fe03c938dbf739e3e0c8aff/family1, priority=12, startTime=1678830940689; duration=0sec 2023-03-14 21:55:41,567 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-14 21:55:41,568 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of df07f4658e3072f4f5757b3e959f3960/family1 in testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:41,568 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp, totalSize=28.9 K 2023-03-14 21:55:41,568 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917428 2023-03-14 21:55:41,569 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:41,569 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920216 2023-03-14 21:55:41,570 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922028 2023-03-14 21:55:41,570 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting fd5a3407a35b420a967e34c18886cba1_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678830917428 2023-03-14 21:55:41,582 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): df07f4658e3072f4f5757b3e959f3960#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:41,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16] to archive 2023-03-14 21:55:41,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:41,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d9e4054fc8664bc08b5a5206e1eeaab9 2023-03-14 21:55:41,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-613d0296f4cb4dfcb66cb3b44c709ce8 2023-03-14 21:55:41,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-d54c5885f76a4a598e92bf77ecbfa6e2 2023-03-14 21:55:41,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-1e8af449b7874e99b8f79f4cd7d62f16 2023-03-14 21:55:41,847 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031487297533c22c4f5fbbe58bc35420a593 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023031487297533c22c4f5fbbe58bc35420a593 2023-03-14 21:55:41,848 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314c300e6ed962e4b8caa3ee41ad1099184 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230314c300e6ed962e4b8caa3ee41ad1099184 2023-03-14 21:55:41,848 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,848 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,877 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/bcd4730f061942e4b9a97c42e4220829 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-14 21:55:41,878 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/bcd4730f061942e4b9a97c42e4220829}] 2023-03-14 21:55:41,879 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/1845b2d9983541c09c06cf7728b9c964 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-14 21:55:41,880 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/1845b2d9983541c09c06cf7728b9c964}] 2023-03-14 21:55:41,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/bcd4730f061942e4b9a97c42e4220829 for inclusion in 068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:41,882 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/1845b2d9983541c09c06cf7728b9c964 for inclusion in f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:41,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-14 21:55:41,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-14 21:55:41,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:41,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314/family1/bcd4730f061942e4b9a97c42e4220829 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__e38ovtftb0hejrmsdpkdn23r9a6ln0pn5j1ekc50bk2efof6n2blgo0ptmod5l0i/family1/bcd4730f061942e4b9a97c42e4220829 2023-03-14 21:55:41,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-14 21:55:41,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-14 21:55:41,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:41,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314/family1/1845b2d9983541c09c06cf7728b9c964 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__baqma528t2fdumdaoojvcl6eoqctg4ui32m2dumshm5gmmlm1vet4ku7tbvjcu3v/family1/1845b2d9983541c09c06cf7728b9c964 2023-03-14 21:55:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__e38ovtftb0hejrmsdpkdn23r9a6ln0pn5j1ekc50bk2efof6n2blgo0ptmod5l0i/family1/bcd4730f061942e4b9a97c42e4220829 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d13d2b60146f4ceaa4355e5efb71e645_SeqId_35_ 2023-03-14 21:55:41,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__e38ovtftb0hejrmsdpkdn23r9a6ln0pn5j1ekc50bk2efof6n2blgo0ptmod5l0i/family1/bcd4730f061942e4b9a97c42e4220829 into 068a37104fe03c938dbf739e3e0c8aff/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d13d2b60146f4ceaa4355e5efb71e645_SeqId_35_ - updating store file list. 2023-03-14 21:55:41,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__baqma528t2fdumdaoojvcl6eoqctg4ui32m2dumshm5gmmlm1vet4ku7tbvjcu3v/family1/1845b2d9983541c09c06cf7728b9c964 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_ 2023-03-14 21:55:41,888 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__baqma528t2fdumdaoojvcl6eoqctg4ui32m2dumshm5gmmlm1vet4ku7tbvjcu3v/family1/1845b2d9983541c09c06cf7728b9c964 into f16657fdf561e3e0eb764d4e08e94891/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_ - updating store file list. 2023-03-14 21:55:41,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d13d2b60146f4ceaa4355e5efb71e645_SeqId_35_ into 068a37104fe03c938dbf739e3e0c8aff/family1 2023-03-14 21:55:41,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__e38ovtftb0hejrmsdpkdn23r9a6ln0pn5j1ekc50bk2efof6n2blgo0ptmod5l0i/family1/bcd4730f061942e4b9a97c42e4220829 into 068a37104fe03c938dbf739e3e0c8aff/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family1/d13d2b60146f4ceaa4355e5efb71e645_SeqId_35_) 2023-03-14 21:55:41,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__e38ovtftb0hejrmsdpkdn23r9a6ln0pn5j1ekc50bk2efof6n2blgo0ptmod5l0i/family1/bcd4730f061942e4b9a97c42e4220829 2023-03-14 21:55:41,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_ into f16657fdf561e3e0eb764d4e08e94891/family1 2023-03-14 21:55:41,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__baqma528t2fdumdaoojvcl6eoqctg4ui32m2dumshm5gmmlm1vet4ku7tbvjcu3v/family1/1845b2d9983541c09c06cf7728b9c964 into f16657fdf561e3e0eb764d4e08e94891/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_) 2023-03-14 21:55:41,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testCompactionWithHFileLink__baqma528t2fdumdaoojvcl6eoqctg4ui32m2dumshm5gmmlm1vet4ku7tbvjcu3v/family1/1845b2d9983541c09c06cf7728b9c964 2023-03-14 21:55:41,892 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:41,893 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:41,895 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303144d63a60422184380b8cd07bf7a27e67a 2023-03-14 21:55:41,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-14 21:55:41,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(1912): f16657fdf561e3e0eb764d4e08e94891/family1 is initiating minor compaction (all files) 2023-03-14 21:55:41,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@f43d7e1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-14 21:55:41,896 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023031466fae3e443db48ce97b2102a48e9ad21 2023-03-14 21:55:41,898 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,898 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303146c138adbb6f6471aa5f611763fcf3af9 2023-03-14 21:55:41,899 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:41,899 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230314f116d1b02598447aacbfc8ab26b20b58 2023-03-14 21:55:41,899 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:41,899 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230314' 2023-03-14 21:55:41,900 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:41,900 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303143cef96eef0954d169aa7e2221cfc64f3 2023-03-14 21:55:41,901 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303146e14b87a6f834448b069274cdbc613e5 2023-03-14 21:55:41,902 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230314ab48a6804d93467893a6f2ead688975b 2023-03-14 21:55:41,903 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230314b87b36badfd64a2ca9f6a15d42cff762 2023-03-14 21:55:41,903 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-14 21:55:41,903 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230314' 2023-03-14 21:55:41,904 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230314 for table='testCompactionWithHFileLink' and column='family1' 2023-03-14 21:55:41,904 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-14 21:55:41,906 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:41,909 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:41,912 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:41,913 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:41,920 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:41,920 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:41,941 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 246) - Thread LEAK? -, OpenFileDescriptor=758 (was 616) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 216) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=3183 (was 3299) 2023-03-14 21:55:41,955 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=758, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=169, AvailableMemoryMB=3183 2023-03-14 21:55:41,960 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-14 21:55:41,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-14 21:55:41,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-14 21:55:41,970 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-14 21:55:41,971 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/.tmp/family2/9c9eeafea0a048e291749653b106adea as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/9c9eeafea0a048e291749653b106adea 2023-03-14 21:55:41,973 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 11 msec 2023-03-14 21:55:41,975 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 068a37104fe03c938dbf739e3e0c8aff/family2 of 068a37104fe03c938dbf739e3e0c8aff into 9c9eeafea0a048e291749653b106adea(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:41,975 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:55:41,975 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff., storeName=068a37104fe03c938dbf739e3e0c8aff/family2, priority=12, startTime=1678830940690; duration=1sec 2023-03-14 21:55:41,975 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-14 21:55:41,975 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f16657fdf561e3e0eb764d4e08e94891/family1 in testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:41,975 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp, totalSize=28.9 K 2023-03-14 21:55:41,976 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917430 2023-03-14 21:55:41,977 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:41,977 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:41,978 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:41,978 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting 538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678830917430 2023-03-14 21:55:41,992 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] throttle.PressureAwareThroughputController(145): f16657fdf561e3e0eb764d4e08e94891#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:42,065 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb] to archive 2023-03-14 21:55:42,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:42,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-14 21:55:42,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-4dfcc1ed30054965a87ed75c1e9bad70 2023-03-14 21:55:42,069 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-5744c93167d04cc8b606b26ae895bfc5 2023-03-14 21:55:42,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-14 21:55:42,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-14 21:55:42,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-ff2fcd642a214f4f9f08e07a2d4af155 2023-03-14 21:55:42,071 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:55:42,071 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-14 21:55:42,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/family2/testCompactionWithHFileLink=068a37104fe03c938dbf739e3e0c8aff-6e98178865674517bf8abef7673555eb 2023-03-14 21:55:42,072 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:55:42,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:42,073 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:42,073 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:42,073 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa empty. 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb empty. 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171 empty. 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:42,074 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:42,074 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-14 21:55:42,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:42,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:42,481 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-14 21:55:42,482 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => c0a565a05ac628117ca5695f343304aa, NAME => 'ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:42,482 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => a9c2e66ef8dbc6ac6ba9e46acd352171, NAME => 'ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:42,483 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => a6dde8759414eded6aef7ffc9c489ceb, NAME => 'ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:55:42,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:42,803 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family1/e86fcc98f9dc46ab95c55241fd805e21 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/e86fcc98f9dc46ab95c55241fd805e21 2023-03-14 21:55:42,807 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in df07f4658e3072f4f5757b3e959f3960/family1 of df07f4658e3072f4f5757b3e959f3960 into e86fcc98f9dc46ab95c55241fd805e21(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:42,808 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:42,808 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678830941097; duration=1sec 2023-03-14 21:55:42,808 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-14 21:55:42,808 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-14 21:55:42,810 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-14 21:55:42,810 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1912): f16657fdf561e3e0eb764d4e08e94891/family2 is initiating minor compaction (all files) 2023-03-14 21:55:42,810 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f16657fdf561e3e0eb764d4e08e94891/family2 in testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:55:42,810 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp, totalSize=20.4 K 2023-03-14 21:55:42,811 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917430 2023-03-14 21:55:42,811 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:42,812 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920217 2023-03-14 21:55:42,812 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922029 2023-03-14 21:55:42,822 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): f16657fdf561e3e0eb764d4e08e94891#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:42,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_] to archive 2023-03-14 21:55:42,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:42,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-104d41f04ac64b5d8d71fcd11b80ee11 2023-03-14 21:55:42,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-c106356df3774a0da464e118374ba796 2023-03-14 21:55:42,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-805864f340094c76a0ec583483d89b0c 2023-03-14 21:55:42,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-5190b039aa1a498aacd63c95ac39fa50 2023-03-14 21:55:42,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family1/fd5a3407a35b420a967e34c18886cba1_SeqId_34_ 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing a9c2e66ef8dbc6ac6ba9e46acd352171, disabling compactions & flushes 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing a6dde8759414eded6aef7ffc9c489ceb, disabling compactions & flushes 2023-03-14 21:55:42,893 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:42,893 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. after waiting 0 ms 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. after waiting 0 ms 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing c0a565a05ac628117ca5695f343304aa, disabling compactions & flushes 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:42,893 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:42,893 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:42,893 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:42,893 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:42,894 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. after waiting 0 ms 2023-03-14 21:55:42,894 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:42,894 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:42,894 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:42,895 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:55:42,896 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678830942896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830942896"}]},"ts":"1678830942896"} 2023-03-14 21:55:42,896 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830942896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830942896"}]},"ts":"1678830942896"} 2023-03-14 21:55:42,896 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830942896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830942896"}]},"ts":"1678830942896"} 2023-03-14 21:55:42,898 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-14 21:55:42,898 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:55:42,899 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830942899"}]},"ts":"1678830942899"} 2023-03-14 21:55:42,900 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-14 21:55:42,902 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0a565a05ac628117ca5695f343304aa, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a6dde8759414eded6aef7ffc9c489ceb, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9c2e66ef8dbc6ac6ba9e46acd352171, ASSIGN}] 2023-03-14 21:55:42,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0a565a05ac628117ca5695f343304aa, ASSIGN 2023-03-14 21:55:42,904 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9c2e66ef8dbc6ac6ba9e46acd352171, ASSIGN 2023-03-14 21:55:42,904 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a6dde8759414eded6aef7ffc9c489ceb, ASSIGN 2023-03-14 21:55:42,905 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0a565a05ac628117ca5695f343304aa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:42,905 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a6dde8759414eded6aef7ffc9c489ceb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:42,905 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9c2e66ef8dbc6ac6ba9e46acd352171, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:55:42,919 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:55:43,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a9c2e66ef8dbc6ac6ba9e46acd352171, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,057 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c0a565a05ac628117ca5695f343304aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,057 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a6dde8759414eded6aef7ffc9c489ceb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830943057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830943057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830943057"}]},"ts":"1678830943057"} 2023-03-14 21:55:43,058 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830943057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830943057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830943057"}]},"ts":"1678830943057"} 2023-03-14 21:55:43,058 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678830943057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830943057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830943057"}]},"ts":"1678830943057"} 2023-03-14 21:55:43,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure a9c2e66ef8dbc6ac6ba9e46acd352171, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:43,060 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure c0a565a05ac628117ca5695f343304aa, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:43,060 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure a6dde8759414eded6aef7ffc9c489ceb, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:55:43,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:43,210 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family1/0ca44b21fac9461c941e9ca39b5a4d9c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/0ca44b21fac9461c941e9ca39b5a4d9c 2023-03-14 21:55:43,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:43,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a6dde8759414eded6aef7ffc9c489ceb, NAME => 'ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-14 21:55:43,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:43,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,214 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in f16657fdf561e3e0eb764d4e08e94891/family1 of f16657fdf561e3e0eb764d4e08e94891 into 0ca44b21fac9461c941e9ca39b5a4d9c(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:43,214 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:43,215 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678830941896; duration=1sec 2023-03-14 21:55:43,215 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,215 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-14 21:55:43,215 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-14 21:55:43,216 DEBUG [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1 2023-03-14 21:55:43,216 DEBUG [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1 2023-03-14 21:55:43,216 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6dde8759414eded6aef7ffc9c489ceb columnFamilyName family1 2023-03-14 21:55:43,217 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-14 21:55:43,217 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1912): df07f4658e3072f4f5757b3e959f3960/family2 is initiating minor compaction (all files) 2023-03-14 21:55:43,217 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2259): Starting compaction of df07f4658e3072f4f5757b3e959f3960/family2 in testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:55:43,217 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp, totalSize=20.4 K 2023-03-14 21:55:43,218 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] regionserver.HStore(310): Store=a6dde8759414eded6aef7ffc9c489ceb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,218 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830917428 2023-03-14 21:55:43,218 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,219 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830919223 2023-03-14 21:55:43,219 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830920216 2023-03-14 21:55:43,220 DEBUG [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2 2023-03-14 21:55:43,220 DEBUG [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2 2023-03-14 21:55:43,220 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830922028 2023-03-14 21:55:43,220 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6dde8759414eded6aef7ffc9c489ceb columnFamilyName family2 2023-03-14 21:55:43,221 INFO [StoreOpener-a6dde8759414eded6aef7ffc9c489ceb-1] regionserver.HStore(310): Store=a6dde8759414eded6aef7ffc9c489ceb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:55:43,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:43,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a6dde8759414eded6aef7ffc9c489ceb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58743619, jitterRate=-0.12465186417102814}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:43,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:43,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb., pid=111, masterSystemTime=1678830943210 2023-03-14 21:55:43,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:43,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:43,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:43,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0a565a05ac628117ca5695f343304aa, NAME => 'ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.', STARTKEY => '', ENDKEY => '1'} 2023-03-14 21:55:43,240 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a6dde8759414eded6aef7ffc9c489ceb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,240 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678830943240"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830943240"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830943240"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830943240"}]},"ts":"1678830943240"} 2023-03-14 21:55:43,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:43,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,242 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] throttle.PressureAwareThroughputController(145): df07f4658e3072f4f5757b3e959f3960#family2#compaction#119 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-14 21:55:43,244 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-03-14 21:55:43,244 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure a6dde8759414eded6aef7ffc9c489ceb, server=jenkins-hbase4.apache.org,35143,1678830821197 in 182 msec 2023-03-14 21:55:43,245 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a6dde8759414eded6aef7ffc9c489ceb, ASSIGN in 342 msec 2023-03-14 21:55:43,247 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,249 DEBUG [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1 2023-03-14 21:55:43,249 DEBUG [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1 2023-03-14 21:55:43,250 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0a565a05ac628117ca5695f343304aa columnFamilyName family1 2023-03-14 21:55:43,251 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] regionserver.HStore(310): Store=c0a565a05ac628117ca5695f343304aa/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,251 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,253 DEBUG [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2 2023-03-14 21:55:43,253 DEBUG [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2 2023-03-14 21:55:43,253 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0a565a05ac628117ca5695f343304aa columnFamilyName family2 2023-03-14 21:55:43,254 INFO [StoreOpener-c0a565a05ac628117ca5695f343304aa-1] regionserver.HStore(310): Store=c0a565a05ac628117ca5695f343304aa/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:43,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c0a565a05ac628117ca5695f343304aa 2023-03-14 21:55:43,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:43,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c0a565a05ac628117ca5695f343304aa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69415381, jitterRate=0.034369781613349915}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:43,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:43,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa., pid=110, masterSystemTime=1678830943210 2023-03-14 21:55:43,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:43,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:43,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a9c2e66ef8dbc6ac6ba9e46acd352171, NAME => 'ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.', STARTKEY => '2', ENDKEY => ''} 2023-03-14 21:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,268 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c0a565a05ac628117ca5695f343304aa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,268 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830943268"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830943268"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830943268"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830943268"}]},"ts":"1678830943268"} 2023-03-14 21:55:43,269 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,270 DEBUG [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1 2023-03-14 21:55:43,270 DEBUG [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1 2023-03-14 21:55:43,271 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9c2e66ef8dbc6ac6ba9e46acd352171 columnFamilyName family1 2023-03-14 21:55:43,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-03-14 21:55:43,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure c0a565a05ac628117ca5695f343304aa, server=jenkins-hbase4.apache.org,35143,1678830821197 in 209 msec 2023-03-14 21:55:43,272 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] regionserver.HStore(310): Store=a9c2e66ef8dbc6ac6ba9e46acd352171/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,272 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,272 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c0a565a05ac628117ca5695f343304aa, ASSIGN in 369 msec 2023-03-14 21:55:43,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_] to archive 2023-03-14 21:55:43,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:43,276 DEBUG [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2 2023-03-14 21:55:43,276 DEBUG [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2 2023-03-14 21:55:43,276 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9c2e66ef8dbc6ac6ba9e46acd352171 columnFamilyName family2 2023-03-14 21:55:43,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-5a71bf3ad2744ecebe89e0ad69ab75ed 2023-03-14 21:55:43,277 INFO [StoreOpener-a9c2e66ef8dbc6ac6ba9e46acd352171-1] regionserver.HStore(310): Store=a9c2e66ef8dbc6ac6ba9e46acd352171/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:55:43,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e92e43a4246c4a4e8dae31c701c4de2f 2023-03-14 21:55:43,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-24a7cc1266384784ac776875b0cce9ff 2023-03-14 21:55:43,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-3d267d78316948f09847a76d93814116 2023-03-14 21:55:43,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:55:43,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family1/538f9c7934a24ff8ac9ec2b2b7714e2d_SeqId_34_ 2023-03-14 21:55:43,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:55:43,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:55:43,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a9c2e66ef8dbc6ac6ba9e46acd352171; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66486128, jitterRate=-0.009279489517211914}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:55:43,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:43,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171., pid=109, masterSystemTime=1678830943210 2023-03-14 21:55:43,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:43,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:43,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a9c2e66ef8dbc6ac6ba9e46acd352171, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:43,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678830943288"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830943288"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830943288"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830943288"}]},"ts":"1678830943288"} 2023-03-14 21:55:43,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-03-14 21:55:43,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure a9c2e66ef8dbc6ac6ba9e46acd352171, server=jenkins-hbase4.apache.org,35143,1678830821197 in 230 msec 2023-03-14 21:55:43,292 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-03-14 21:55:43,292 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9c2e66ef8dbc6ac6ba9e46acd352171, ASSIGN in 388 msec 2023-03-14 21:55:43,293 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:55:43,293 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830943293"}]},"ts":"1678830943293"} 2023-03-14 21:55:43,294 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-14 21:55:43,297 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:55:43,298 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2280 sec 2023-03-14 21:55:44,046 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/.tmp/family2/b1c9e0e6c9d14d0993ad3b65c072cd81 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/b1c9e0e6c9d14d0993ad3b65c072cd81 2023-03-14 21:55:44,051 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f16657fdf561e3e0eb764d4e08e94891/family2 of f16657fdf561e3e0eb764d4e08e94891 into b1c9e0e6c9d14d0993ad3b65c072cd81(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:55:44,051 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891., storeName=f16657fdf561e3e0eb764d4e08e94891/family2, priority=12, startTime=1678830940831; duration=1sec 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. because compaction request was cancelled 2023-03-14 21:55:44,051 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-14 21:55:44,052 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-14 21:55:44,052 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-14 21:55:44,052 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. because compaction request was cancelled 2023-03-14 21:55:44,065 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6] to archive 2023-03-14 21:55:44,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:44,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-e8d4a324c2e54e328bd67e2b2377a1ea 2023-03-14 21:55:44,068 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/.tmp/family2/492f766e40564036b575e197b9c528ea as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/492f766e40564036b575e197b9c528ea 2023-03-14 21:55:44,069 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-6de3f715832e4389b1a7cc6d9dc660c4 2023-03-14 21:55:44,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-9024cea521624ccb89d9220fbb802bab 2023-03-14 21:55:44,071 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/family2/testCompactionWithHFileLink=f16657fdf561e3e0eb764d4e08e94891-66f769e5545e44679e833945f42b4ac6 2023-03-14 21:55:44,072 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in df07f4658e3072f4f5757b3e959f3960/family2 of df07f4658e3072f4f5757b3e959f3960 into 492f766e40564036b575e197b9c528ea(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:44,072 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:55:44,072 INFO [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960., storeName=df07f4658e3072f4f5757b3e959f3960/family2, priority=12, startTime=1678830940776; duration=0sec 2023-03-14 21:55:44,072 DEBUG [RS:0;jenkins-hbase4:35143-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:55:44,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d] to archive 2023-03-14 21:55:44,170 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:44,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-edd24bca9b49439b9bd49f6f9b0743af 2023-03-14 21:55:44,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-a881deaf5aee415990f4d1bfab3bd1c8 2023-03-14 21:55:44,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-f1d2d5054f0b4eb984516db4af8dc85c 2023-03-14 21:55:44,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/family2/testCompactionWithHFileLink=df07f4658e3072f4f5757b3e959f3960-1659b63d0821455080073630bd919a4d 2023-03-14 21:55:44,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-14 21:55:44,176 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-14 21:55:44,182 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:44,183 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:44,183 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:44,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:44,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-14 21:55:44,187 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-14 21:55:44,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:44,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-14 21:55:44,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-14 21:55:44,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:44,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:44,188 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:44,188 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:44,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:44,189 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:44,189 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:44,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:44,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-14 21:55:44,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-14 21:55:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-14 21:55:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:44,190 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-14 21:55:44,190 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:44,190 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. started... 2023-03-14 21:55:44,193 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:44,193 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a6dde8759414eded6aef7ffc9c489ceb 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:44,193 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:44,193 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. started... 2023-03-14 21:55:44,193 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:44,193 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:44,193 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a9c2e66ef8dbc6ac6ba9e46acd352171 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:44,193 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. started... 2023-03-14 21:55:44,193 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c0a565a05ac628117ca5695f343304aa 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:55:44,211 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031419a9ed59140f4147b5473fa82bc22a12 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023031419a9ed59140f4147b5473fa82bc22a12 2023-03-14 21:55:44,609 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303148921f29cb4c24f03ae348eb2a723f3fc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303148921f29cb4c24f03ae348eb2a723f3fc 2023-03-14 21:55:44,613 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303141264ee44b0b34ef380cf404f767f656f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303141264ee44b0b34ef380cf404f767f656f 2023-03-14 21:55:44,614 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/b3ab69a1c42247f9a8ad6b380f59f259 2023-03-14 21:55:44,617 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/8796202a97904a04ac2494bc739122f2 2023-03-14 21:55:44,618 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb 2023-03-14 21:55:44,629 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303148725f92046ac480a9039305dc9c40e7a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303148725f92046ac480a9039305dc9c40e7a 2023-03-14 21:55:44,633 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/d579dacdadbf4870b5e4c4537ee6a0b0 2023-03-14 21:55:44,635 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314fde1e6c7e08b4c549341d1db1c2244e6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230314fde1e6c7e08b4c549341d1db1c2244e6 2023-03-14 21:55:44,638 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/b3ab69a1c42247f9a8ad6b380f59f259 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259 2023-03-14 21:55:44,642 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:44,643 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/d579dacdadbf4870b5e4c4537ee6a0b0 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0 2023-03-14 21:55:44,647 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:44,648 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c0a565a05ac628117ca5695f343304aa in 455ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:44,648 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-14 21:55:44,649 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:44,649 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:44,649 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:45,033 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314fa99aa1b8b654b1886b9079de1f738e1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230314fa99aa1b8b654b1886b9079de1f738e1 2023-03-14 21:55:45,058 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/3024166b75c3431b950fdae1560b96fb 2023-03-14 21:55:45,062 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/8796202a97904a04ac2494bc739122f2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2 2023-03-14 21:55:45,066 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:45,067 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/3024166b75c3431b950fdae1560b96fb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb 2023-03-14 21:55:45,070 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:45,071 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a6dde8759414eded6aef7ffc9c489ceb in 878ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:45,072 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:45,072 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:45,439 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/823b332bc1394d958c4a0f65616c9802 2023-03-14 21:55:45,443 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb 2023-03-14 21:55:45,446 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb, entries=4, sequenceid=5, filesize=5.4 K 2023-03-14 21:55:45,447 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/823b332bc1394d958c4a0f65616c9802 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802 2023-03-14 21:55:45,450 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:55:45,451 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a9c2e66ef8dbc6ac6ba9e46acd352171 in 1258ms, sequenceid=5, compaction requested=false 2023-03-14 21:55:45,451 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:45,451 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:45,451 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:45,451 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:45,451 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,451 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-14 21:55:45,451 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-14 21:55:45,453 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,453 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:45,453 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:45,453 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:45,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-14 21:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52d843ae[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:45,454 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-14 21:55:45,454 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,455 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,455 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-14 21:55:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-14 21:55:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:45,456 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,456 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-14 21:55:45,458 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-14 21:55:45,460 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-14 21:55:45,460 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:45,460 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:45,460 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-14 21:55:45,460 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:45,461 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,461 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,462 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:45,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:45,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:45,462 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,463 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,465 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,465 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:45,465 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:45,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:45,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-14 21:55:45,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-14 21:55:45,465 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:45,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:45,465 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:45,465 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:45,466 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:45,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:45,566 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:45,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:45,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:45,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:45,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:45,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:45,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:45,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-14 21:55:45,572 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-14 21:55:45,573 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:45,573 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-14 21:55:45,573 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-14 21:55:45,573 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,573 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:45,574 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,574 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:45,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:45,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:45,575 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:45,575 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-14 21:55:45,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-14 21:55:45,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-14 21:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-14 21:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. started... 2023-03-14 21:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:45,576 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a6dde8759414eded6aef7ffc9c489ceb 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:45,576 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. started... 2023-03-14 21:55:45,576 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. started... 2023-03-14 21:55:45,576 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c0a565a05ac628117ca5695f343304aa 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:45,576 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a9c2e66ef8dbc6ac6ba9e46acd352171 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:55:45,990 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303144ffaaff797cd44939ca0bd9ae37f7216 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303144ffaaff797cd44939ca0bd9ae37f7216 2023-03-14 21:55:45,990 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314cd3cfcb5569243ed80d95c5c6de4a9f7 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230314cd3cfcb5569243ed80d95c5c6de4a9f7 2023-03-14 21:55:45,990 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303148e28b11395d34a9abc62b7f336231ecf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303148e28b11395d34a9abc62b7f336231ecf 2023-03-14 21:55:46,005 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe 2023-03-14 21:55:46,406 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/40305158030a4c3eaeb00d37c20b710b 2023-03-14 21:55:46,406 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/ce5a018b2180414baa18254b2cdf67bc 2023-03-14 21:55:46,419 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314e84af9603723484bbaebe09ffe760f54 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230314e84af9603723484bbaebe09ffe760f54 2023-03-14 21:55:46,421 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031498bb67ebfb56409a91d3f7597751f957 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023031498bb67ebfb56409a91d3f7597751f957 2023-03-14 21:55:46,421 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314428bc561401d45dbb71ad5b546a9b563 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230314428bc561401d45dbb71ad5b546a9b563 2023-03-14 21:55:46,432 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/738cf95818b94e438b4d6fe0346c0fdf 2023-03-14 21:55:46,436 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe 2023-03-14 21:55:46,439 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:46,440 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/738cf95818b94e438b4d6fe0346c0fdf as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf 2023-03-14 21:55:46,443 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:46,444 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a6dde8759414eded6aef7ffc9c489ceb in 868ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:46,444 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:46,444 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:46,832 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/82728d6305954606aa3a63a1e2a37635 2023-03-14 21:55:46,836 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/ce5a018b2180414baa18254b2cdf67bc as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc 2023-03-14 21:55:46,839 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:46,840 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/82728d6305954606aa3a63a1e2a37635 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635 2023-03-14 21:55:46,843 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:46,844 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a9c2e66ef8dbc6ac6ba9e46acd352171 in 1268ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:46,844 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:46,844 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:46,844 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:46,846 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/0d1bee610c6a41dda26ff7e90eb37079 2023-03-14 21:55:46,849 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/40305158030a4c3eaeb00d37c20b710b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b 2023-03-14 21:55:46,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-14 21:55:46,853 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/0d1bee610c6a41dda26ff7e90eb37079 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079 2023-03-14 21:55:46,856 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:55:46,857 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c0a565a05ac628117ca5695f343304aa in 1281ms, sequenceid=9, compaction requested=false 2023-03-14 21:55:46,857 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:46,857 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:46,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:46,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:46,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-14 21:55:46,858 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-14 21:55:46,861 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:46,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:46,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:46,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:46,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-14 21:55:46,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@325229d5[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:46,863 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-14 21:55:46,863 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,864 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-14 21:55:46,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-14 21:55:46,864 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:46,864 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,864 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:46,868 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-14 21:55:46,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:46,869 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,869 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:46,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:46,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:46,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-14 21:55:46,871 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-14 21:55:46,871 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:46,871 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:46,871 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-14 21:55:46,871 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:46,873 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,873 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,873 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:46,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:46,874 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:46,875 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:46,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:46,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,877 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:46,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-14 21:55:46,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:46,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:46,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:46,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:46,877 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:46,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:46,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:46,977 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:46,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:46,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:46,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:46,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:46,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:46,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:46,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-14 21:55:46,983 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-14 21:55:46,983 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:46,983 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-14 21:55:46,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-14 21:55:46,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,984 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:46,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,985 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:46,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:46,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:46,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:46,985 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:46,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:46,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:46,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-14 21:55:46,986 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-14 21:55:46,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-14 21:55:46,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-14 21:55:46,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:46,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. started... 2023-03-14 21:55:46,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:46,987 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. started... 2023-03-14 21:55:46,987 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. started... 2023-03-14 21:55:46,987 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a6dde8759414eded6aef7ffc9c489ceb 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:46,987 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c0a565a05ac628117ca5695f343304aa 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:46,987 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a9c2e66ef8dbc6ac6ba9e46acd352171 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:55:47,400 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303148cf12b34cf08408eb96d3e0689e97841 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303148cf12b34cf08408eb96d3e0689e97841 2023-03-14 21:55:47,400 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303140e825a31b3fa44658c64aaab570e2839 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303140e825a31b3fa44658c64aaab570e2839 2023-03-14 21:55:47,400 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303147f90558ebcae4ea4b4c2e7991c1c3592 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303147f90558ebcae4ea4b4c2e7991c1c3592 2023-03-14 21:55:47,805 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/6e40427d480d4c44a33b339622ed5477 2023-03-14 21:55:47,805 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/cdf82157e2b543b4b0e09c1faef1e73f 2023-03-14 21:55:47,805 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/d7eb32475f0f4cd5a029354b00eb3481 2023-03-14 21:55:48,219 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031467edd74032474514930dac294cc5e448 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023031467edd74032474514930dac294cc5e448 2023-03-14 21:55:48,219 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314b49ee8d0e6b7445384477580a557e551 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230314b49ee8d0e6b7445384477580a557e551 2023-03-14 21:55:48,220 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303148af609adc8d148ffbb078f654bbecd93 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303148af609adc8d148ffbb078f654bbecd93 2023-03-14 21:55:48,625 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/05672a7fa8534c3e85c0cf420763b435 2023-03-14 21:55:48,625 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/c62bab0dccd04e9cae1d88399b265a5c 2023-03-14 21:55:48,625 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/b0eb9a20c892415394b5056ed602a1f2 2023-03-14 21:55:48,630 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/cdf82157e2b543b4b0e09c1faef1e73f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f 2023-03-14 21:55:48,630 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/6e40427d480d4c44a33b339622ed5477 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477 2023-03-14 21:55:48,630 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/d7eb32475f0f4cd5a029354b00eb3481 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481 2023-03-14 21:55:48,633 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:48,633 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:48,634 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477, entries=4, sequenceid=13, filesize=5.4 K 2023-03-14 21:55:48,634 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/b0eb9a20c892415394b5056ed602a1f2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2 2023-03-14 21:55:48,634 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/c62bab0dccd04e9cae1d88399b265a5c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c 2023-03-14 21:55:48,635 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/05672a7fa8534c3e85c0cf420763b435 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435 2023-03-14 21:55:48,638 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:48,638 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:48,639 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a9c2e66ef8dbc6ac6ba9e46acd352171 in 1652ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:48,639 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:55:48,639 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:48,639 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a6dde8759414eded6aef7ffc9c489ceb in 1652ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:48,639 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:48,639 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:48,639 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:48,639 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c0a565a05ac628117ca5695f343304aa in 1652ms, sequenceid=13, compaction requested=true 2023-03-14 21:55:48,640 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:48,640 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-14 21:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-14 21:55:48,641 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,641 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,642 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-14 21:55:48,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41e85fd1[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:48,643 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-14 21:55:48,643 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,645 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,645 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-14 21:55:48,645 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-14 21:55:48,645 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:48,645 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,645 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:48,646 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-14 21:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:48,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-14 21:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:48,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:48,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-14 21:55:48,649 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-14 21:55:48,649 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:48,649 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:48,649 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-14 21:55:48,649 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:48,650 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,650 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:48,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,650 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:48,651 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:48,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:48,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:48,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:48,652 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:48,654 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:48,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-14 21:55:48,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-14 21:55:48,654 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:48,655 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:48,715 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:55:48,739 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:55:48,739 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d67da39 to 127.0.0.1:58882 2023-03-14 21:55:48,739 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:48,739 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:55:48,740 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e943a5b to 127.0.0.1:58882 2023-03-14 21:55:48,740 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:55:48,755 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:48,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:48,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:48,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:48,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:55:48,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:48,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:48,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-14 21:55:48,761 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-14 21:55:48,761 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:48,761 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-14 21:55:48,762 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-14 21:55:48,762 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,762 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:48,763 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,763 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:48,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:48,763 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:48,764 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:48,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:48,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-14 21:55:48,765 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-14 21:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-14 21:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:48,765 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-14 21:55:48,765 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:48,765 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. started... 2023-03-14 21:55:48,765 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a6dde8759414eded6aef7ffc9c489ceb 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:48,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:48,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:48,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:48,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. started... 2023-03-14 21:55:48,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:48,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. started... 2023-03-14 21:55:48,768 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a9c2e66ef8dbc6ac6ba9e46acd352171 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:48,768 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c0a565a05ac628117ca5695f343304aa 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:55:49,185 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314a6d249c43ff7460ba2befe02e8e98c1c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230314a6d249c43ff7460ba2befe02e8e98c1c 2023-03-14 21:55:49,189 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314e5adf31514444b1d9b4567dfbc5956a2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230314e5adf31514444b1d9b4567dfbc5956a2 2023-03-14 21:55:49,190 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314ed75991e2f6342d589c5b6f6ced1b4ea to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230314ed75991e2f6342d589c5b6f6ced1b4ea 2023-03-14 21:55:49,590 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/e5c82484e6664dfea2be5b03c5b464ea 2023-03-14 21:55:49,594 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/6ff42f4869824fec9eeec6b88d444d60 2023-03-14 21:55:49,594 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/37eea8759ccf4cdea6ea8d1e42332cf8 2023-03-14 21:55:50,005 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314576a74880d324c58a4084fa4f1cc74cd to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230314576a74880d324c58a4084fa4f1cc74cd 2023-03-14 21:55:50,010 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/61ee1cebeec14a01b51c298ee1bbec1b 2023-03-14 21:55:50,010 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314fffa31e9f9f4470a93bfabdd8181662e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230314fffa31e9f9f4470a93bfabdd8181662e 2023-03-14 21:55:50,011 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314e274ab8ed20b48e2855239694fc96e7d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230314e274ab8ed20b48e2855239694fc96e7d 2023-03-14 21:55:50,013 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/e5c82484e6664dfea2be5b03c5b464ea as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea 2023-03-14 21:55:50,015 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/c5d02251d5944aa4a71794db5f4751fb 2023-03-14 21:55:50,017 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:50,018 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/61ee1cebeec14a01b51c298ee1bbec1b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b 2023-03-14 21:55:50,018 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/6ff42f4869824fec9eeec6b88d444d60 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60 2023-03-14 21:55:50,021 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:50,022 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a6dde8759414eded6aef7ffc9c489ceb in 1257ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:50,022 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:50,022 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:50,022 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:50,022 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:50,023 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/c5d02251d5944aa4a71794db5f4751fb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb 2023-03-14 21:55:50,026 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:50,026 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c0a565a05ac628117ca5695f343304aa in 1258ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:50,026 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:50,026 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:50,026 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:50,416 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/4c032b715d4846e3a0b26e033ceb3ed2 2023-03-14 21:55:50,420 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/37eea8759ccf4cdea6ea8d1e42332cf8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8 2023-03-14 21:55:50,423 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8, entries=4, sequenceid=17, filesize=5.4 K 2023-03-14 21:55:50,424 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/4c032b715d4846e3a0b26e033ceb3ed2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2 2023-03-14 21:55:50,428 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:55:50,428 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a9c2e66ef8dbc6ac6ba9e46acd352171 in 1660ms, sequenceid=17, compaction requested=true 2023-03-14 21:55:50,428 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:50,428 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:50,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:50,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:50,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-14 21:55:50,429 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-14 21:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,430 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:50,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:50,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:50,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:50,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:50,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-14 21:55:50,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@681f40d5[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:50,432 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-14 21:55:50,432 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,434 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,434 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-14 21:55:50,434 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-14 21:55:50,435 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:50,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,435 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:50,436 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-14 21:55:50,436 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,436 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:50,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:50,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:50,436 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-14 21:55:50,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:50,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:50,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-14 21:55:50,438 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-14 21:55:50,438 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:50,438 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:50,438 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-14 21:55:50,438 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:50,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,439 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:50,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,440 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:50,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:50,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,441 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:50,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:50,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,443 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:50,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:50,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-14 21:55:50,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:50,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:50,443 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:50,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:50,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:50,544 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:50,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:50,552 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:50,558 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:50,566 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:50,567 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:50,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:55:50,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-14 21:55:50,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-14 21:55:50,593 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-14 21:55:50,593 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:50,593 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-14 21:55:50,593 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-14 21:55:50,593 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,593 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:50,595 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:50,595 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:50,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:50,596 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:50,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:50,596 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:55:50,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:50,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:55:50,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:50,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-14 21:55:50,597 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-14 21:55:50,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-14 21:55:50,597 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-14 21:55:50,597 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:55:50,597 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. started... 2023-03-14 21:55:50,597 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:55:50,597 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:55:50,597 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a6dde8759414eded6aef7ffc9c489ceb 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. started... 2023-03-14 21:55:50,597 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. started... 2023-03-14 21:55:50,597 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a9c2e66ef8dbc6ac6ba9e46acd352171 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:50,598 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c0a565a05ac628117ca5695f343304aa 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:55:51,421 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/57345808c0cc49f9b6a8da91f09e39b0 2023-03-14 21:55:51,424 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57345808c0cc49f9b6a8da91f09e39b0 2023-03-14 21:55:51,425 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/8d618caead8a4c08a1841bddab2ce31e 2023-03-14 21:55:51,426 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/7c0d4de20e6f4bd295b0e6cc82868902 2023-03-14 21:55:51,429 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8d618caead8a4c08a1841bddab2ce31e 2023-03-14 21:55:51,429 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c0d4de20e6f4bd295b0e6cc82868902 2023-03-14 21:55:51,842 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/07a7ddccf9e84f0f96584c27b3721581 2023-03-14 21:55:51,845 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07a7ddccf9e84f0f96584c27b3721581 2023-03-14 21:55:51,846 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/7c0d4de20e6f4bd295b0e6cc82868902 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902 2023-03-14 21:55:51,849 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c0d4de20e6f4bd295b0e6cc82868902 2023-03-14 21:55:51,849 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902, entries=3, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:51,850 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/07a7ddccf9e84f0f96584c27b3721581 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581 2023-03-14 21:55:51,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07a7ddccf9e84f0f96584c27b3721581 2023-03-14 21:55:51,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:51,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c0a565a05ac628117ca5695f343304aa in 1255ms, sequenceid=23, compaction requested=true 2023-03-14 21:55:51,854 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:51,854 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:51,854 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:55:52,237 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/08169d2a7d3349698e7ddaa379096de4 2023-03-14 21:55:52,240 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08169d2a7d3349698e7ddaa379096de4 2023-03-14 21:55:52,241 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/57345808c0cc49f9b6a8da91f09e39b0 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0 2023-03-14 21:55:52,242 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/b37574fc3a1b4fceaa2bba124f57388d 2023-03-14 21:55:52,245 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57345808c0cc49f9b6a8da91f09e39b0 2023-03-14 21:55:52,245 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0, entries=3, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:52,245 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37574fc3a1b4fceaa2bba124f57388d 2023-03-14 21:55:52,245 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/08169d2a7d3349698e7ddaa379096de4 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4 2023-03-14 21:55:52,246 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/8d618caead8a4c08a1841bddab2ce31e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e 2023-03-14 21:55:52,249 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8d618caead8a4c08a1841bddab2ce31e 2023-03-14 21:55:52,249 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e, entries=3, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:52,249 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08169d2a7d3349698e7ddaa379096de4 2023-03-14 21:55:52,249 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:52,249 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/b37574fc3a1b4fceaa2bba124f57388d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d 2023-03-14 21:55:52,250 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a6dde8759414eded6aef7ffc9c489ceb in 1653ms, sequenceid=23, compaction requested=true 2023-03-14 21:55:52,250 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:52,250 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:55:52,253 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37574fc3a1b4fceaa2bba124f57388d 2023-03-14 21:55:52,253 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:55:52,254 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a9c2e66ef8dbc6ac6ba9e46acd352171 in 1657ms, sequenceid=23, compaction requested=true 2023-03-14 21:55:52,254 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:52,254 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:52,254 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:55:52,254 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:55:52,254 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,254 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-14 21:55:52,254 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-14 21:55:52,256 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,256 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:52,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,257 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:52,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-14 21:55:52,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b8a9cd0[Count = 0] remaining members to acquire global barrier 2023-03-14 21:55:52,258 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-14 21:55:52,258 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,259 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,259 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-14 21:55:52,259 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-14 21:55:52,259 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,259 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:55:52,259 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:55:52,269 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-14 21:55:52,269 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:52,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:52,269 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-14 21:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-14 21:55:52,272 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-14 21:55:52,272 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:55:52,272 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:55:52,272 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-14 21:55:52,272 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:55:52,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:52,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:55:52,274 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:55:52,275 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:55:52,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-14 21:55:52,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:52,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:55:52,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:55:52,277 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-14 21:55:52,277 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-14 21:55:52,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-14 21:55:52,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:55:52,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-14 21:55:52,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:55:52,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:55:52,278 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-14 21:55:52,278 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:55:52,378 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:55:52,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:55:52,379 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:52,380 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): a6dde8759414eded6aef7ffc9c489ceb/family1 is initiating major compaction (all files) 2023-03-14 21:55:52,380 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:52,380 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:52,380 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of a6dde8759414eded6aef7ffc9c489ceb/family1 in ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:52,380 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp, totalSize=26.6 K 2023-03-14 21:55:52,381 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 8796202a97904a04ac2494bc739122f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944183 2023-03-14 21:55:52,381 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting bc6431c6ec5249ffaf4dec7d14cdb8fe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:52,382 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d7eb32475f0f4cd5a029354b00eb3481, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946979 2023-03-14 21:55:52,382 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting e5c82484e6664dfea2be5b03c5b464ea, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:52,382 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 57345808c0cc49f9b6a8da91f09e39b0, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:52,396 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): a6dde8759414eded6aef7ffc9c489ceb#family1#compaction#150 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:52,809 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303147a5b4ddb0be74787a81ea769bfd7cf37_del 2023-03-14 21:55:52,809 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303147a5b4ddb0be74787a81ea769bfd7cf37_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303147a5b4ddb0be74787a81ea769bfd7cf37_del 2023-03-14 21:55:53,622 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37b9fb22cf5543c39006ab75384ed371 2023-03-14 21:55:53,623 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family1/37b9fb22cf5543c39006ab75384ed371 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/37b9fb22cf5543c39006ab75384ed371 2023-03-14 21:55:53,627 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37b9fb22cf5543c39006ab75384ed371 2023-03-14 21:55:53,628 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a6dde8759414eded6aef7ffc9c489ceb/family1 of a6dde8759414eded6aef7ffc9c489ceb into 37b9fb22cf5543c39006ab75384ed371(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:53,628 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:53,628 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:53,628 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): a6dde8759414eded6aef7ffc9c489ceb/family2 is initiating major compaction (all files) 2023-03-14 21:55:53,628 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:53,628 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:53,629 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of a6dde8759414eded6aef7ffc9c489ceb/family2 in ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:55:53,629 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp, totalSize=25.3 K 2023-03-14 21:55:53,630 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 3024166b75c3431b950fdae1560b96fb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944183 2023-03-14 21:55:53,630 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 738cf95818b94e438b4d6fe0346c0fdf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:53,631 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting c62bab0dccd04e9cae1d88399b265a5c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946979 2023-03-14 21:55:53,631 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 61ee1cebeec14a01b51c298ee1bbec1b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:53,631 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 08169d2a7d3349698e7ddaa379096de4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:53,640 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): a6dde8759414eded6aef7ffc9c489ceb#family2#compaction#151 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:53,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0] to archive 2023-03-14 21:55:53,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:53,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/8796202a97904a04ac2494bc739122f2 2023-03-14 21:55:53,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/bc6431c6ec5249ffaf4dec7d14cdb8fe 2023-03-14 21:55:53,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/d7eb32475f0f4cd5a029354b00eb3481 2023-03-14 21:55:53,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/e5c82484e6664dfea2be5b03c5b464ea 2023-03-14 21:55:53,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/57345808c0cc49f9b6a8da91f09e39b0 2023-03-14 21:55:54,048 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023031483fa9025f4b14071a10c12f1cb801232_del 2023-03-14 21:55:54,048 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031483fa9025f4b14071a10c12f1cb801232_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023031483fa9025f4b14071a10c12f1cb801232_del 2023-03-14 21:55:54,862 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19e0db35e15f4015aaaa6bf2b6a5c2f8 2023-03-14 21:55:54,862 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/.tmp/family2/19e0db35e15f4015aaaa6bf2b6a5c2f8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/19e0db35e15f4015aaaa6bf2b6a5c2f8 2023-03-14 21:55:54,865 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19e0db35e15f4015aaaa6bf2b6a5c2f8 2023-03-14 21:55:54,866 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a6dde8759414eded6aef7ffc9c489ceb/family2 of a6dde8759414eded6aef7ffc9c489ceb into 19e0db35e15f4015aaaa6bf2b6a5c2f8(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:54,866 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:54,866 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:54,866 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): a9c2e66ef8dbc6ac6ba9e46acd352171/family1 is initiating major compaction (all files) 2023-03-14 21:55:54,866 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:54,866 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:54,867 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of a9c2e66ef8dbc6ac6ba9e46acd352171/family1 in ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:54,867 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp, totalSize=26.6 K 2023-03-14 21:55:54,867 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 1d809a1bb7a24a5893b07fd2c2a0a9bb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944183 2023-03-14 21:55:54,867 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting ce5a018b2180414baa18254b2cdf67bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:54,867 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting cdf82157e2b543b4b0e09c1faef1e73f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946980 2023-03-14 21:55:54,868 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 37eea8759ccf4cdea6ea8d1e42332cf8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:54,868 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 8d618caead8a4c08a1841bddab2ce31e, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:54,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4] to archive 2023-03-14 21:55:54,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:54,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/3024166b75c3431b950fdae1560b96fb 2023-03-14 21:55:54,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/738cf95818b94e438b4d6fe0346c0fdf 2023-03-14 21:55:54,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/c62bab0dccd04e9cae1d88399b265a5c 2023-03-14 21:55:54,875 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): a9c2e66ef8dbc6ac6ba9e46acd352171#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:54,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/61ee1cebeec14a01b51c298ee1bbec1b 2023-03-14 21:55:54,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family2/08169d2a7d3349698e7ddaa379096de4 2023-03-14 21:55:55,283 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303144f6e2b4ca79c4a08bf87e53a767fc22a_del 2023-03-14 21:55:55,284 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303144f6e2b4ca79c4a08bf87e53a767fc22a_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303144f6e2b4ca79c4a08bf87e53a767fc22a_del 2023-03-14 21:55:56,096 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e88c7ceb9524ddd89edfb81f9e14679 2023-03-14 21:55:56,097 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family1/3e88c7ceb9524ddd89edfb81f9e14679 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/3e88c7ceb9524ddd89edfb81f9e14679 2023-03-14 21:55:56,101 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e88c7ceb9524ddd89edfb81f9e14679 2023-03-14 21:55:56,101 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9c2e66ef8dbc6ac6ba9e46acd352171/family1 of a9c2e66ef8dbc6ac6ba9e46acd352171 into 3e88c7ceb9524ddd89edfb81f9e14679(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:56,102 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:56,102 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:56,102 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): a9c2e66ef8dbc6ac6ba9e46acd352171/family2 is initiating major compaction (all files) 2023-03-14 21:55:56,102 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:56,102 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:56,103 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of a9c2e66ef8dbc6ac6ba9e46acd352171/family2 in ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:55:56,103 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp, totalSize=25.3 K 2023-03-14 21:55:56,103 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 823b332bc1394d958c4a0f65616c9802, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944183 2023-03-14 21:55:56,103 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 82728d6305954606aa3a63a1e2a37635, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:56,104 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting b0eb9a20c892415394b5056ed602a1f2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946980 2023-03-14 21:55:56,104 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 4c032b715d4846e3a0b26e033ceb3ed2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:56,104 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting b37574fc3a1b4fceaa2bba124f57388d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:56,111 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): a9c2e66ef8dbc6ac6ba9e46acd352171#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:56,165 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e] to archive 2023-03-14 21:55:56,166 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:56,167 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/1d809a1bb7a24a5893b07fd2c2a0a9bb 2023-03-14 21:55:56,168 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/ce5a018b2180414baa18254b2cdf67bc 2023-03-14 21:55:56,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/cdf82157e2b543b4b0e09c1faef1e73f 2023-03-14 21:55:56,170 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/37eea8759ccf4cdea6ea8d1e42332cf8 2023-03-14 21:55:56,171 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/8d618caead8a4c08a1841bddab2ce31e 2023-03-14 21:55:56,519 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230314e35137661fc644d4b7fa73dd4a2fcccd_del 2023-03-14 21:55:56,519 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314e35137661fc644d4b7fa73dd4a2fcccd_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230314e35137661fc644d4b7fa73dd4a2fcccd_del 2023-03-14 21:55:57,332 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4bf0cf2f976c431f9d9e9c3c2ca59ed3 2023-03-14 21:55:57,333 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/.tmp/family2/4bf0cf2f976c431f9d9e9c3c2ca59ed3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4bf0cf2f976c431f9d9e9c3c2ca59ed3 2023-03-14 21:55:57,336 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4bf0cf2f976c431f9d9e9c3c2ca59ed3 2023-03-14 21:55:57,337 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9c2e66ef8dbc6ac6ba9e46acd352171/family2 of a9c2e66ef8dbc6ac6ba9e46acd352171 into 4bf0cf2f976c431f9d9e9c3c2ca59ed3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:57,337 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:57,337 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:57,337 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): c0a565a05ac628117ca5695f343304aa/family1 is initiating major compaction (all files) 2023-03-14 21:55:57,338 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:57,338 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:57,338 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of c0a565a05ac628117ca5695f343304aa/family1 in ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:57,338 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp, totalSize=26.6 K 2023-03-14 21:55:57,338 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting b3ab69a1c42247f9a8ad6b380f59f259, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944182 2023-03-14 21:55:57,339 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 40305158030a4c3eaeb00d37c20b710b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:57,339 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 6e40427d480d4c44a33b339622ed5477, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946979 2023-03-14 21:55:57,340 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 6ff42f4869824fec9eeec6b88d444d60, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:57,340 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 7c0d4de20e6f4bd295b0e6cc82868902, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:57,349 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): c0a565a05ac628117ca5695f343304aa#family1#compaction#154 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:57,367 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d] to archive 2023-03-14 21:55:57,368 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:57,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/823b332bc1394d958c4a0f65616c9802 2023-03-14 21:55:57,371 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/82728d6305954606aa3a63a1e2a37635 2023-03-14 21:55:57,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b0eb9a20c892415394b5056ed602a1f2 2023-03-14 21:55:57,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/4c032b715d4846e3a0b26e033ceb3ed2 2023-03-14 21:55:57,374 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family2/b37574fc3a1b4fceaa2bba124f57388d 2023-03-14 21:55:57,756 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031416cf2f377a894d608ab71b2841e5a9bb_del 2023-03-14 21:55:57,756 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031416cf2f377a894d608ab71b2841e5a9bb_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023031416cf2f377a894d608ab71b2841e5a9bb_del 2023-03-14 21:55:57,769 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95d5bbe2d9a2473da8c3d7b21e0bfdb2 2023-03-14 21:55:57,769 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family1/95d5bbe2d9a2473da8c3d7b21e0bfdb2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/95d5bbe2d9a2473da8c3d7b21e0bfdb2 2023-03-14 21:55:57,773 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95d5bbe2d9a2473da8c3d7b21e0bfdb2 2023-03-14 21:55:57,773 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0a565a05ac628117ca5695f343304aa/family1 of c0a565a05ac628117ca5695f343304aa into 95d5bbe2d9a2473da8c3d7b21e0bfdb2(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:55:57,774 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:57,774 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:55:57,774 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): c0a565a05ac628117ca5695f343304aa/family2 is initiating major compaction (all files) 2023-03-14 21:55:57,774 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:55:57,774 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:55:57,776 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of c0a565a05ac628117ca5695f343304aa/family2 in ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:55:57,776 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp, totalSize=25.3 K 2023-03-14 21:55:57,776 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d579dacdadbf4870b5e4c4537ee6a0b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830944182 2023-03-14 21:55:57,777 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 0d1bee610c6a41dda26ff7e90eb37079, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830945568 2023-03-14 21:55:57,777 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 05672a7fa8534c3e85c0cf420763b435, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830946979 2023-03-14 21:55:57,777 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting c5d02251d5944aa4a71794db5f4751fb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830948757 2023-03-14 21:55:57,778 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 07a7ddccf9e84f0f96584c27b3721581, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:55:57,785 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): c0a565a05ac628117ca5695f343304aa#family2#compaction#155 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:55:57,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902] to archive 2023-03-14 21:55:57,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:57,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/b3ab69a1c42247f9a8ad6b380f59f259 2023-03-14 21:55:57,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/40305158030a4c3eaeb00d37c20b710b 2023-03-14 21:55:57,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6e40427d480d4c44a33b339622ed5477 2023-03-14 21:55:57,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6ff42f4869824fec9eeec6b88d444d60 2023-03-14 21:55:57,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/7c0d4de20e6f4bd295b0e6cc82868902 2023-03-14 21:55:58,194 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303148aec0963471b4729ae33e8ec620449a6_del 2023-03-14 21:55:58,194 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303148aec0963471b4729ae33e8ec620449a6_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303148aec0963471b4729ae33e8ec620449a6_del 2023-03-14 21:55:59,007 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47413e3ce63a4c3b87c8ab132bcd5a70 2023-03-14 21:55:59,008 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/.tmp/family2/47413e3ce63a4c3b87c8ab132bcd5a70 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/47413e3ce63a4c3b87c8ab132bcd5a70 2023-03-14 21:55:59,011 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47413e3ce63a4c3b87c8ab132bcd5a70 2023-03-14 21:55:59,012 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0a565a05ac628117ca5695f343304aa/family2 of c0a565a05ac628117ca5695f343304aa into 47413e3ce63a4c3b87c8ab132bcd5a70(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:55:59,012 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:59,015 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:59,018 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:59,020 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:59,021 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:59,026 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:55:59,038 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-14 21:55:59,040 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023031416cf2f377a894d608ab71b2841e5a9bb_del 2023-03-14 21:55:59,043 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303147a5b4ddb0be74787a81ea769bfd7cf37_del 2023-03-14 21:55:59,045 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303144f6e2b4ca79c4a08bf87e53a767fc22a_del 2023-03-14 21:55:59,045 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,045 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(139): Connect 0x19affb6f to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:55:59,049 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@293644a6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:55:59,051 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,051 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,051 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,065 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581] to archive 2023-03-14 21:55:59,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:59,068 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/d579dacdadbf4870b5e4c4537ee6a0b0 2023-03-14 21:55:59,069 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/0d1bee610c6a41dda26ff7e90eb37079 2023-03-14 21:55:59,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/05672a7fa8534c3e85c0cf420763b435 2023-03-14 21:55:59,071 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/c5d02251d5944aa4a71794db5f4751fb 2023-03-14 21:55:59,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family2/07a7ddccf9e84f0f96584c27b3721581 2023-03-14 21:55:59,877 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314b728e1a909514735947dbc398a86a2bc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230314b728e1a909514735947dbc398a86a2bc 2023-03-14 21:55:59,878 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303145ae720d965e84adb856c47a9d3e04ce9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303145ae720d965e84adb856c47a9d3e04ce9 2023-03-14 21:55:59,878 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,878 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,884 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303142e40394bc0764da18ca3b4569ee24f36 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303142e40394bc0764da18ca3b4569ee24f36 2023-03-14 21:55:59,885 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,891 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:55:59,899 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54202, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:55:59,905 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:55:59,913 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59936, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:55:59,936 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230314/family1/2c7db837902644479511a9c12318163d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-14 21:55:59,938 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/family1/b8ff65302ee04532b0d175aabe80f35d first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-14 21:55:59,939 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230314/family1/2c7db837902644479511a9c12318163d}] 2023-03-14 21:55:59,940 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230314/family1/254148c292f54b149619e5e3ed5b3add first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-14 21:55:59,940 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/family1/b8ff65302ee04532b0d175aabe80f35d}] 2023-03-14 21:55:59,941 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230314/family1/254148c292f54b149619e5e3ed5b3add}] 2023-03-14 21:55:59,950 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/family1/b8ff65302ee04532b0d175aabe80f35d for inclusion in c0a565a05ac628117ca5695f343304aa/family1 2023-03-14 21:55:59,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230314/family1/2c7db837902644479511a9c12318163d for inclusion in a6dde8759414eded6aef7ffc9c489ceb/family1 2023-03-14 21:55:59,950 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230314/family1/254148c292f54b149619e5e3ed5b3add for inclusion in a9c2e66ef8dbc6ac6ba9e46acd352171/family1 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last=1 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:55:59,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:55:59,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230314/family1/254148c292f54b149619e5e3ed5b3add to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__cohedl9986etq7rul77f0f6tquqcheq8tvklui8fuhpv4bpca208gdg2ennjjkjv/family1/254148c292f54b149619e5e3ed5b3add 2023-03-14 21:55:59,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230314/family1/b8ff65302ee04532b0d175aabe80f35d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__beuqe8h87g14qudf1fiei3vit8fvdhffrolprp63slbvc6g9gpgh95r1fjbgsec5/family1/b8ff65302ee04532b0d175aabe80f35d 2023-03-14 21:55:59,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230314/family1/2c7db837902644479511a9c12318163d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__33kdn69vip015n1q5iuvkfnvrm2kgjvadkpgbs9lb4a5d656sr15mu5gmgku13j5/family1/2c7db837902644479511a9c12318163d 2023-03-14 21:55:59,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__cohedl9986etq7rul77f0f6tquqcheq8tvklui8fuhpv4bpca208gdg2ennjjkjv/family1/254148c292f54b149619e5e3ed5b3add as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/9a5b881354ad4ba8ba92c8dc3bbdd819_SeqId_28_ 2023-03-14 21:55:59,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__33kdn69vip015n1q5iuvkfnvrm2kgjvadkpgbs9lb4a5d656sr15mu5gmgku13j5/family1/2c7db837902644479511a9c12318163d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/945f83acf91343a88c1114c307b4f8a4_SeqId_28_ 2023-03-14 21:55:59,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__cohedl9986etq7rul77f0f6tquqcheq8tvklui8fuhpv4bpca208gdg2ennjjkjv/family1/254148c292f54b149619e5e3ed5b3add into a9c2e66ef8dbc6ac6ba9e46acd352171/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/9a5b881354ad4ba8ba92c8dc3bbdd819_SeqId_28_ - updating store file list. 2023-03-14 21:55:59,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__33kdn69vip015n1q5iuvkfnvrm2kgjvadkpgbs9lb4a5d656sr15mu5gmgku13j5/family1/2c7db837902644479511a9c12318163d into a6dde8759414eded6aef7ffc9c489ceb/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/945f83acf91343a88c1114c307b4f8a4_SeqId_28_ - updating store file list. 2023-03-14 21:55:59,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__beuqe8h87g14qudf1fiei3vit8fvdhffrolprp63slbvc6g9gpgh95r1fjbgsec5/family1/b8ff65302ee04532b0d175aabe80f35d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6c56e5b194d04948be0cc21e5ff7195c_SeqId_28_ 2023-03-14 21:55:59,958 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__beuqe8h87g14qudf1fiei3vit8fvdhffrolprp63slbvc6g9gpgh95r1fjbgsec5/family1/b8ff65302ee04532b0d175aabe80f35d into c0a565a05ac628117ca5695f343304aa/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6c56e5b194d04948be0cc21e5ff7195c_SeqId_28_ - updating store file list. 2023-03-14 21:55:59,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/945f83acf91343a88c1114c307b4f8a4_SeqId_28_ into a6dde8759414eded6aef7ffc9c489ceb/family1 2023-03-14 21:55:59,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__33kdn69vip015n1q5iuvkfnvrm2kgjvadkpgbs9lb4a5d656sr15mu5gmgku13j5/family1/2c7db837902644479511a9c12318163d into a6dde8759414eded6aef7ffc9c489ceb/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/family1/945f83acf91343a88c1114c307b4f8a4_SeqId_28_) 2023-03-14 21:55:59,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/9a5b881354ad4ba8ba92c8dc3bbdd819_SeqId_28_ into a9c2e66ef8dbc6ac6ba9e46acd352171/family1 2023-03-14 21:55:59,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__cohedl9986etq7rul77f0f6tquqcheq8tvklui8fuhpv4bpca208gdg2ennjjkjv/family1/254148c292f54b149619e5e3ed5b3add into a9c2e66ef8dbc6ac6ba9e46acd352171/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/family1/9a5b881354ad4ba8ba92c8dc3bbdd819_SeqId_28_) 2023-03-14 21:55:59,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__cohedl9986etq7rul77f0f6tquqcheq8tvklui8fuhpv4bpca208gdg2ennjjkjv/family1/254148c292f54b149619e5e3ed5b3add 2023-03-14 21:55:59,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__33kdn69vip015n1q5iuvkfnvrm2kgjvadkpgbs9lb4a5d656sr15mu5gmgku13j5/family1/2c7db837902644479511a9c12318163d 2023-03-14 21:55:59,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6c56e5b194d04948be0cc21e5ff7195c_SeqId_28_ into c0a565a05ac628117ca5695f343304aa/family1 2023-03-14 21:55:59,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__beuqe8h87g14qudf1fiei3vit8fvdhffrolprp63slbvc6g9gpgh95r1fjbgsec5/family1/b8ff65302ee04532b0d175aabe80f35d into c0a565a05ac628117ca5695f343304aa/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/family1/6c56e5b194d04948be0cc21e5ff7195c_SeqId_28_) 2023-03-14 21:55:59,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__ns_testMinorCompaction__beuqe8h87g14qudf1fiei3vit8fvdhffrolprp63slbvc6g9gpgh95r1fjbgsec5/family1/b8ff65302ee04532b0d175aabe80f35d 2023-03-14 21:55:59,964 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,965 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,965 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:59,966 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:59,966 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,966 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:55:59,969 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023031419a9ed59140f4147b5473fa82bc22a12 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023031419a9ed59140f4147b5473fa82bc22a12 2023-03-14 21:55:59,969 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-14 21:55:59,969 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303141264ee44b0b34ef380cf404f767f656f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303141264ee44b0b34ef380cf404f767f656f 2023-03-14 21:55:59,969 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230314' 2023-03-14 21:55:59,969 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-14 21:55:59,970 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230314' 2023-03-14 21:55:59,970 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,970 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,970 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303148921f29cb4c24f03ae348eb2a723f3fc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303148921f29cb4c24f03ae348eb2a723f3fc 2023-03-14 21:55:59,970 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-14 21:55:59,970 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230314' 2023-03-14 21:55:59,971 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230314 for table='ns:testMinorCompaction' and column='family1' 2023-03-14 21:55:59,971 INFO [Listener at localhost/45975] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-14 21:55:59,973 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:59,975 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:59,979 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:55:59,980 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:55:59,998 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=249 (was 285), OpenFileDescriptor=623 (was 758), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 225) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 169), AvailableMemoryMB=3181 (was 3183) 2023-03-14 21:56:00,011 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=623, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=169, AvailableMemoryMB=3181 2023-03-14 21:56:00,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-14 21:56:00,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:00,016 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:56:00,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-14 21:56:00,017 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:56:00,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:00,018 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:00,018 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474 empty. 2023-03-14 21:56:00,019 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:00,019 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-14 21:56:00,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:00,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:00,426 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-14 21:56:00,428 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 23a9524a7f137f65decd5c4c40a09474, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:56:00,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 23a9524a7f137f65decd5c4c40a09474, disabling compactions & flushes 2023-03-14 21:56:00,833 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. after waiting 0 ms 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:00,833 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:00,833 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:00,835 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:56:00,836 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830960836"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830960836"}]},"ts":"1678830960836"} 2023-03-14 21:56:00,837 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-14 21:56:00,838 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:56:00,838 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830960838"}]},"ts":"1678830960838"} 2023-03-14 21:56:00,839 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-14 21:56:00,843 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=23a9524a7f137f65decd5c4c40a09474, ASSIGN}] 2023-03-14 21:56:00,844 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=23a9524a7f137f65decd5c4c40a09474, ASSIGN 2023-03-14 21:56:00,845 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=23a9524a7f137f65decd5c4c40a09474, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:56:00,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=23a9524a7f137f65decd5c4c40a09474, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:00,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830960996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830960996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830960996"}]},"ts":"1678830960996"} 2023-03-14 21:56:00,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 23a9524a7f137f65decd5c4c40a09474, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:56:01,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:01,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:01,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 23a9524a7f137f65decd5c4c40a09474, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.', STARTKEY => '', ENDKEY => ''} 2023-03-14 21:56:01,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:01,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,153 INFO [StoreOpener-23a9524a7f137f65decd5c4c40a09474-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,155 DEBUG [StoreOpener-23a9524a7f137f65decd5c4c40a09474-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:01,155 DEBUG [StoreOpener-23a9524a7f137f65decd5c4c40a09474-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:01,155 INFO [StoreOpener-23a9524a7f137f65decd5c4c40a09474-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23a9524a7f137f65decd5c4c40a09474 columnFamilyName family1 2023-03-14 21:56:01,156 INFO [StoreOpener-23a9524a7f137f65decd5c4c40a09474-1] regionserver.HStore(310): Store=23a9524a7f137f65decd5c4c40a09474/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:01,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:01,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:56:01,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 23a9524a7f137f65decd5c4c40a09474; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69438720, jitterRate=0.034717559814453125}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-14 21:56:01,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:01,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474., pid=114, masterSystemTime=1678830961149 2023-03-14 21:56:01,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:01,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:01,163 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=23a9524a7f137f65decd5c4c40a09474, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:01,163 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678830961163"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830961163"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830961163"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830961163"}]},"ts":"1678830961163"} 2023-03-14 21:56:01,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-14 21:56:01,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 23a9524a7f137f65decd5c4c40a09474, server=jenkins-hbase4.apache.org,35143,1678830821197 in 167 msec 2023-03-14 21:56:01,166 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-14 21:56:01,166 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=23a9524a7f137f65decd5c4c40a09474, ASSIGN in 322 msec 2023-03-14 21:56:01,167 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:56:01,167 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830961167"}]},"ts":"1678830961167"} 2023-03-14 21:56:01,168 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-14 21:56:01,170 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:56:01,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1560 sec 2023-03-14 21:56:02,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-14 21:56:02,123 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-14 21:56:02,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:02,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,161 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:02,161 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:02,161 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:02,161 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:02,161 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,161 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,163 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:02,163 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:02,163 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,163 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:02,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:02,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:02,168 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:02,168 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:02,168 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:02,168 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:02,169 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-14 21:56:02,194 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d1426d7db25b429ba1e7d85d85cf43ff to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d1426d7db25b429ba1e7d85d85cf43ff 2023-03-14 21:56:02,198 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/61662981f0264e5db60fc81d31113208 2023-03-14 21:56:02,203 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/61662981f0264e5db60fc81d31113208 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/61662981f0264e5db60fc81d31113208 2023-03-14 21:56:02,207 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/61662981f0264e5db60fc81d31113208, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-14 21:56:02,208 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 39ms, sequenceid=5, compaction requested=false 2023-03-14 21:56:02,208 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:02,209 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:02,209 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:02,209 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:02,212 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,212 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:02,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,213 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:02,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:02,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:02,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19c5a53e[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:02,215 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:02,215 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,216 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,217 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:02,217 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:02,217 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:02,217 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,217 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:02,218 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,218 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:02,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:02,218 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:02,218 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:02,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:02,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:02,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:02,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:02,221 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:02,221 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:02,221 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:02,221 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,221 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,223 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:02,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:02,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:02,223 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:02,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:02,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:02,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:02,228 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:02,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:02,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:02,229 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:02,229 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,229 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,229 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:02,229 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:02,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:02,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:02,329 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:02,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:02,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:02,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:02,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,335 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:02,335 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:02,335 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:02,335 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:02,336 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,336 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,339 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:02,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:02,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:02,339 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:02,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:02,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:02,340 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:02,341 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:02,341 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:02,341 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:02,341 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:02,341 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-14 21:56:02,752 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b284c6c81a5846ecb0ed3520708e20aa to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b284c6c81a5846ecb0ed3520708e20aa 2023-03-14 21:56:03,156 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/a7c8a28e6c9d42658023413dc50b8ba2 2023-03-14 21:56:03,160 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/a7c8a28e6c9d42658023413dc50b8ba2 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2 2023-03-14 21:56:03,164 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2, entries=1, sequenceid=9, filesize=5.0 K 2023-03-14 21:56:03,164 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 823ms, sequenceid=9, compaction requested=false 2023-03-14 21:56:03,165 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:03,165 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:03,166 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:03,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:03,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:03,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@540f5d0f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:03,168 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:03,168 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,169 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:03,169 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,169 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:03,171 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:03,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:03,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:03,174 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:03,174 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:03,174 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:03,174 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,174 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:03,175 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,175 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,175 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:03,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:03,176 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,177 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:03,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:03,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:03,179 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:03,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:03,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:03,279 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:03,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:03,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:03,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:03,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,286 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:03,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:03,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:03,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:03,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,291 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:03,291 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:03,291 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:03,291 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:03,293 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:03,293 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:03,293 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:03,703 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201a2f8c757231c46adbe0f16839241ddbf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201a2f8c757231c46adbe0f16839241ddbf 2023-03-14 21:56:04,107 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/f4681729029743438c6c2c5017644183 2023-03-14 21:56:04,111 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/f4681729029743438c6c2c5017644183 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183 2023-03-14 21:56:04,114 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183, entries=1, sequenceid=13, filesize=4.9 K 2023-03-14 21:56:04,115 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 822ms, sequenceid=13, compaction requested=true 2023-03-14 21:56:04,115 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:04,115 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:04,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:04,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:04,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:04,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:04,117 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,117 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:04,117 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,117 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:04,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:04,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4604a6b3[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:04,119 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:04,119 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,120 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,120 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:04,120 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:04,120 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:04,120 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,120 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:04,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:04,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:04,123 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:04,124 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:04,124 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:04,124 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,124 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:04,126 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,126 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:04,126 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:04,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:04,126 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,127 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,129 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,129 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:04,129 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,129 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:04,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:04,129 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:04,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:04,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:04,130 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:04,230 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:04,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:04,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:04,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:04,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,236 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:04,236 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:04,236 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:04,236 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:04,236 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,236 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,237 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,237 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:04,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:04,238 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:04,238 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:04,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:04,239 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:04,239 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:04,239 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:04,239 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:04,239 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:04,239 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:04,239 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:04,239 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:04,649 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dd9bb777b412486aa038addad5e2567f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd9bb777b412486aa038addad5e2567f 2023-03-14 21:56:05,054 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/32de3779e66347c8bbb96beaf2653957 2023-03-14 21:56:05,059 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/32de3779e66347c8bbb96beaf2653957 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957 2023-03-14 21:56:05,063 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957, entries=1, sequenceid=17, filesize=4.9 K 2023-03-14 21:56:05,064 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 825ms, sequenceid=17, compaction requested=true 2023-03-14 21:56:05,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:05,064 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:05,064 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:05,066 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,066 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,066 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,066 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:05,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7957debf[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:05,067 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:05,067 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,070 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,070 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:05,070 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:05,070 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:05,070 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,070 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:05,071 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,071 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:05,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,071 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:05,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,071 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:05,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:05,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:05,073 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:05,073 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:05,073 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:05,073 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,073 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:05,074 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,075 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:05,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,075 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,076 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,078 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:05,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:05,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:05,078 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,079 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:05,079 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:05,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:05,179 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:05,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:05,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:05,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:05,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,185 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:05,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:05,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:05,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:05,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,185 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,187 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:05,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:05,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:05,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,187 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:05,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,188 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:05,188 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:05,189 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:05,189 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:05,189 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:05,189 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:05,189 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:05,189 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:05,199 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512282c42ce16a1fe4605a4431546ea9769bc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282c42ce16a1fe4605a4431546ea9769bc 2023-03-14 21:56:05,203 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/c8c527d38291421dbfadaecffa857118 2023-03-14 21:56:05,206 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/c8c527d38291421dbfadaecffa857118 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118 2023-03-14 21:56:05,209 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118, entries=1, sequenceid=21, filesize=4.9 K 2023-03-14 21:56:05,210 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 21ms, sequenceid=21, compaction requested=true 2023-03-14 21:56:05,210 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:05,210 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:05,210 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:05,212 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,212 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,212 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,212 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:05,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d0f68fe[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:05,213 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:05,213 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,214 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,214 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:05,214 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:05,214 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:05,214 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,214 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:05,216 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:05,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:05,218 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:05,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:05,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:05,218 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,218 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:05,220 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:05,221 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:05,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:05,221 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,222 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,227 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:05,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:05,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:05,228 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:05,228 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:05,228 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:05,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:05,328 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:05,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:05,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:05,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:05,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,334 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:05,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:05,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:05,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:05,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,334 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,335 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:05,335 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:05,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:05,336 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:05,336 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:05,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:05,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:05,337 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:05,337 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:05,337 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:05,337 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:05,337 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:05,337 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:05,337 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:05,337 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:05,747 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231026ba769d49d4185889bee88ea7e0749 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231026ba769d49d4185889bee88ea7e0749 2023-03-14 21:56:06,152 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/59637ecedd474f7496cec2e3eb98982f 2023-03-14 21:56:06,155 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/59637ecedd474f7496cec2e3eb98982f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f 2023-03-14 21:56:06,159 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f, entries=1, sequenceid=25, filesize=4.9 K 2023-03-14 21:56:06,159 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 822ms, sequenceid=25, compaction requested=true 2023-03-14 21:56:06,159 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:06,160 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:06,160 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:06,160 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:06,160 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,160 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:06,160 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:06,162 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,162 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,162 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,163 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:06,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:06,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63c6b375[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:06,164 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:06,164 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,165 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:06,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:06,165 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:06,165 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,165 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:06,166 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:06,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,166 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:06,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:06,169 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:06,169 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:06,169 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:06,169 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,169 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:06,171 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,171 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,171 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:06,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,172 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,173 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:06,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:06,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:06,178 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:06,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:06,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:06,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:06,178 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:06,179 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:06,279 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:06,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:06,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:06,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:06,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,285 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:06,285 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:06,285 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:06,285 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:06,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,286 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,287 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:06,287 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:06,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:06,287 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,287 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:06,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:06,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:06,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:06,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:06,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:06,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:06,288 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:06,288 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:06,288 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:06,289 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:06,690 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:56:06,699 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101654bc53afb294ccc87f9ed5406e3cc5c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101654bc53afb294ccc87f9ed5406e3cc5c 2023-03-14 21:56:06,705 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/663671ee4ed044ae9bf27e51a0fa2bab 2023-03-14 21:56:06,708 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/663671ee4ed044ae9bf27e51a0fa2bab as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab 2023-03-14 21:56:06,712 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab, entries=1, sequenceid=29, filesize=4.9 K 2023-03-14 21:56:06,712 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 423ms, sequenceid=29, compaction requested=true 2023-03-14 21:56:06,712 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:06,712 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:06,712 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:06,713 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:06,713 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,713 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:06,713 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:06,714 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,714 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,714 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,714 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:06,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@94a080[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:06,716 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:06,716 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,717 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,717 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:06,717 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:06,717 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,717 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:06,717 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:06,719 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:06,719 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,719 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,720 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:06,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:06,723 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:06,723 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:06,723 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:06,723 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,723 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:06,724 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,724 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:06,724 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:06,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:06,725 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:06,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,728 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:06,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,731 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,731 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:06,731 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:06,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:06,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:06,731 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:06,732 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:06,732 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:06,832 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:06,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:06,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:06,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:06,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,839 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:06,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:06,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:06,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:06,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,840 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,841 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,841 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:06,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:06,841 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:06,841 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:06,843 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:06,843 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:06,843 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:06,853 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ac32164b29ec49e481ebd955c2d0cf4e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103ac32164b29ec49e481ebd955c2d0cf4e 2023-03-14 21:56:07,258 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/4780f3d553ea44bc8c52514e282681b6 2023-03-14 21:56:07,262 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/4780f3d553ea44bc8c52514e282681b6 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6 2023-03-14 21:56:07,265 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6, entries=1, sequenceid=33, filesize=4.9 K 2023-03-14 21:56:07,266 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 423ms, sequenceid=33, compaction requested=true 2023-03-14 21:56:07,266 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:07,266 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:07,269 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,269 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:07,270 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,270 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:07,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f1340b[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:07,271 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:07,271 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,273 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:07,273 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:07,273 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,273 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:07,274 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:07,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:07,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:07,276 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:07,276 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:07,276 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:07,276 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,276 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:07,278 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:07,278 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:07,278 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,279 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:07,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:07,285 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:07,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:07,285 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,286 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:07,286 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:07,286 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,286 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:07,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:07,286 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,386 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:07,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:07,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:07,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:07,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,392 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:07,392 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:07,392 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:07,392 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:07,392 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,392 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,393 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:07,393 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:07,394 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:07,394 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,395 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:07,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:07,395 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:07,395 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:07,395 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:07,807 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073436f6da4fd2447a9912140cdada5c41 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509073436f6da4fd2447a9912140cdada5c41 2023-03-14 21:56:08,212 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/8c50e3fc65964a349bb36bfe62ac9b56 2023-03-14 21:56:08,216 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/8c50e3fc65964a349bb36bfe62ac9b56 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56 2023-03-14 21:56:08,219 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56, entries=1, sequenceid=37, filesize=4.9 K 2023-03-14 21:56:08,220 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 825ms, sequenceid=37, compaction requested=true 2023-03-14 21:56:08,220 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:08,220 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:08,220 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:08,221 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:08,221 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,221 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:08,221 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:08,222 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,222 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,223 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,223 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:08,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@710fec93[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:08,224 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:08,224 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,225 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,225 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:08,225 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:08,225 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:08,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,225 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:08,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:08,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:08,227 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,227 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:08,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:08,230 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:08,230 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:08,230 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:08,230 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,230 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:08,232 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,232 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,232 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:08,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,233 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:08,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:08,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:08,235 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:08,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-14 21:56:08,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:08,235 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,236 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:08,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:08,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:08,236 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,236 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,236 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,336 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:08,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:08,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:08,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:08,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,342 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:08,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:08,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-14 21:56:08,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-14 21:56:08,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,342 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,343 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,343 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:08,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:08,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:08,344 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,344 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-14 21:56:08,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-14 21:56:08,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:08,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-14 21:56:08,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-14 21:56:08,345 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:08,345 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:08,345 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. started... 2023-03-14 21:56:08,345 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 23a9524a7f137f65decd5c4c40a09474 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-14 21:56:08,355 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120df5f0733ccc04ed5bfe0c04677d40708 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df5f0733ccc04ed5bfe0c04677d40708 2023-03-14 21:56:08,760 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/793f5bdaa04c4934bf9a337f106037ef 2023-03-14 21:56:08,764 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/793f5bdaa04c4934bf9a337f106037ef as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef 2023-03-14 21:56:08,767 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef, entries=1, sequenceid=41, filesize=4.9 K 2023-03-14 21:56:08,768 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 23a9524a7f137f65decd5c4c40a09474 in 423ms, sequenceid=41, compaction requested=true 2023-03-14 21:56:08,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:08,768 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:08,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-14 21:56:08,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-14 21:56:08,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-14 21:56:08,768 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-14 21:56:08,771 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,771 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,772 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,772 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-14 21:56:08,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2784b4dc[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:08,773 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-14 21:56:08,773 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,774 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,774 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-14 21:56:08,775 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-14 21:56:08,775 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,775 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:08,775 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:08,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-14 21:56:08,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:08,776 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,776 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-14 21:56:08,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:08,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-14 21:56:08,778 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-14 21:56:08,778 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:08,778 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:08,778 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,778 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:08,780 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,780 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:08,780 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:08,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:08,780 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:08,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,781 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,784 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,784 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:08,784 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,785 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-14 21:56:08,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:08,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:08,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:08,785 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:08,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:08,885 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:08,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:08,887 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-14 21:56:08,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:56:08,890 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:56:08,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:56:08,890 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831568890 2023-03-14 21:56:08,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-14 21:56:08,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-14 21:56:08,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:08,892 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x7614f55a to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:56:08,896 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d77417c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:56:08,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:09,721 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103afd87d80efcb4ab49e52505955f6fc0e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103afd87d80efcb4ab49e52505955f6fc0e 2023-03-14 21:56:09,722 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:09,734 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:56:09,736 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45098, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:56:09,742 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:56:09,743 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55914, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:56:09,751 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/e7f86dc1be924344b4e57c6a874db803 first=Optional[r5] last=Optional[r6] 2023-03-14 21:56:09,751 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/e7f86dc1be924344b4e57c6a874db803}] 2023-03-14 21:56:09,758 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/e7f86dc1be924344b4e57c6a874db803 for inclusion in 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:09,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-14 21:56:09,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:56:09,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:09,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/e7f86dc1be924344b4e57c6a874db803 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__29doddltr88e2u9itnirqui2g8j69jvepffo7k136a56p8ef0cngfiqvreq05nfn/family1/e7f86dc1be924344b4e57c6a874db803 2023-03-14 21:56:09,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__29doddltr88e2u9itnirqui2g8j69jvepffo7k136a56p8ef0cngfiqvreq05nfn/family1/e7f86dc1be924344b4e57c6a874db803 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_ 2023-03-14 21:56:09,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__29doddltr88e2u9itnirqui2g8j69jvepffo7k136a56p8ef0cngfiqvreq05nfn/family1/e7f86dc1be924344b4e57c6a874db803 into 23a9524a7f137f65decd5c4c40a09474/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_ - updating store file list. 2023-03-14 21:56:09,766 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_ into 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:09,766 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__29doddltr88e2u9itnirqui2g8j69jvepffo7k136a56p8ef0cngfiqvreq05nfn/family1/e7f86dc1be924344b4e57c6a874db803 into 23a9524a7f137f65decd5c4c40a09474/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_) 2023-03-14 21:56:09,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__29doddltr88e2u9itnirqui2g8j69jvepffo7k136a56p8ef0cngfiqvreq05nfn/family1/e7f86dc1be924344b4e57c6a874db803 2023-03-14 21:56:09,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-14 21:56:09,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-14 21:56:09,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(1912): 23a9524a7f137f65decd5c4c40a09474/family1 is initiating minor compaction 2023-03-14 21:56:09,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@72b8843a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:56:09,769 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 23a9524a7f137f65decd5c4c40a09474/family1 in testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:09,769 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp, totalSize=49.7 K 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting a7c8a28e6c9d42658023413dc50b8ba2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting f4681729029743438c6c2c5017644183, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 32de3779e66347c8bbb96beaf2653957, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting c8c527d38291421dbfadaecffa857118, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 59637ecedd474f7496cec2e3eb98982f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 663671ee4ed044ae9bf27e51a0fa2bab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 4780f3d553ea44bc8c52514e282681b6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-14 21:56:09,769 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 8c50e3fc65964a349bb36bfe62ac9b56, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-14 21:56:09,770 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 793f5bdaa04c4934bf9a337f106037ef, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-14 21:56:09,770 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] compactions.Compactor(207): Compacting 4f05055f06ad4779a54218609f56cd1f_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-14 21:56:09,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:09,771 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:09,773 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101654bc53afb294ccc87f9ed5406e3cc5c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101654bc53afb294ccc87f9ed5406e3cc5c 2023-03-14 21:56:09,774 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103ac32164b29ec49e481ebd955c2d0cf4e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103ac32164b29ec49e481ebd955c2d0cf4e 2023-03-14 21:56:09,774 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-14 21:56:09,774 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-14 21:56:09,775 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:09,775 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:09,799 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] throttle.PressureAwareThroughputController(145): 23a9524a7f137f65decd5c4c40a09474#family1#compaction#166 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-14 21:56:10,228 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313989658c2e90437494c52eb289d58e97 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512313989658c2e90437494c52eb289d58e97 2023-03-14 21:56:10,229 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:10,250 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bfc543154c104108b03d9a5cd356b081 first=Optional[r1] last=Optional[r4] 2023-03-14 21:56:10,250 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bfc543154c104108b03d9a5cd356b081}] 2023-03-14 21:56:10,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bfc543154c104108b03d9a5cd356b081 for inclusion in 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:10,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-14 21:56:10,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:56:10,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:10,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bfc543154c104108b03d9a5cd356b081 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6dfabbqppgmt69ovuendhkhhdjuk4f1k80tk1j1fcms8jm34lbo6svmsc2kqbvk2/family1/bfc543154c104108b03d9a5cd356b081 2023-03-14 21:56:10,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6dfabbqppgmt69ovuendhkhhdjuk4f1k80tk1j1fcms8jm34lbo6svmsc2kqbvk2/family1/bfc543154c104108b03d9a5cd356b081 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/83e85827c56649ec85e8c46d009e95bd_SeqId_46_ 2023-03-14 21:56:10,257 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6dfabbqppgmt69ovuendhkhhdjuk4f1k80tk1j1fcms8jm34lbo6svmsc2kqbvk2/family1/bfc543154c104108b03d9a5cd356b081 into 23a9524a7f137f65decd5c4c40a09474/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/83e85827c56649ec85e8c46d009e95bd_SeqId_46_ - updating store file list. 2023-03-14 21:56:10,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/83e85827c56649ec85e8c46d009e95bd_SeqId_46_ into 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:10,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6dfabbqppgmt69ovuendhkhhdjuk4f1k80tk1j1fcms8jm34lbo6svmsc2kqbvk2/family1/bfc543154c104108b03d9a5cd356b081 into 23a9524a7f137f65decd5c4c40a09474/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/83e85827c56649ec85e8c46d009e95bd_SeqId_46_) 2023-03-14 21:56:10,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__6dfabbqppgmt69ovuendhkhhdjuk4f1k80tk1j1fcms8jm34lbo6svmsc2kqbvk2/family1/bfc543154c104108b03d9a5cd356b081 2023-03-14 21:56:10,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:10,263 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:10,265 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201a2f8c757231c46adbe0f16839241ddbf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201a2f8c757231c46adbe0f16839241ddbf 2023-03-14 21:56:10,266 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd9bb777b412486aa038addad5e2567f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205dd9bb777b412486aa038addad5e2567f 2023-03-14 21:56:10,267 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282c42ce16a1fe4605a4431546ea9769bc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512282c42ce16a1fe4605a4431546ea9769bc 2023-03-14 21:56:10,268 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231026ba769d49d4185889bee88ea7e0749 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231026ba769d49d4185889bee88ea7e0749 2023-03-14 21:56:10,268 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-14 21:56:10,268 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-14 21:56:10,268 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:10,269 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:10,311 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ee10d2c82b3e40d8be80cf0bbd43ea88 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130ee10d2c82b3e40d8be80cf0bbd43ea88 2023-03-14 21:56:10,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:10,341 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5760263abc764d16830c2a50603b0ae9 first=Optional[r00] last=Optional[r8] 2023-03-14 21:56:10,341 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5760263abc764d16830c2a50603b0ae9}] 2023-03-14 21:56:10,343 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5760263abc764d16830c2a50603b0ae9 for inclusion in 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:10,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-14 21:56:10,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last= 2023-03-14 21:56:10,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:10,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5760263abc764d16830c2a50603b0ae9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__fn2idrrtisqb7fsm1h5evhkf0i47pbrn50l2s0bs78b4cajko7jtb2rdrgj5rnj3/family1/5760263abc764d16830c2a50603b0ae9 2023-03-14 21:56:10,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__fn2idrrtisqb7fsm1h5evhkf0i47pbrn50l2s0bs78b4cajko7jtb2rdrgj5rnj3/family1/5760263abc764d16830c2a50603b0ae9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/522577d99d644f63b15001caec6700f5_SeqId_48_ 2023-03-14 21:56:10,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__fn2idrrtisqb7fsm1h5evhkf0i47pbrn50l2s0bs78b4cajko7jtb2rdrgj5rnj3/family1/5760263abc764d16830c2a50603b0ae9 into 23a9524a7f137f65decd5c4c40a09474/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/522577d99d644f63b15001caec6700f5_SeqId_48_ - updating store file list. 2023-03-14 21:56:10,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/522577d99d644f63b15001caec6700f5_SeqId_48_ into 23a9524a7f137f65decd5c4c40a09474/family1 2023-03-14 21:56:10,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__fn2idrrtisqb7fsm1h5evhkf0i47pbrn50l2s0bs78b4cajko7jtb2rdrgj5rnj3/family1/5760263abc764d16830c2a50603b0ae9 into 23a9524a7f137f65decd5c4c40a09474/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/522577d99d644f63b15001caec6700f5_SeqId_48_) 2023-03-14 21:56:10,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMinorCompactionWithMonthlyPolicy__fn2idrrtisqb7fsm1h5evhkf0i47pbrn50l2s0bs78b4cajko7jtb2rdrgj5rnj3/family1/5760263abc764d16830c2a50603b0ae9 2023-03-14 21:56:10,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.SortedCompactionPolicy(75): Selecting compaction from 13 store files, 10 compacting, 2 eligible, 16 blocking 2023-03-14 21:56:10,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-14 21:56:10,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-14 21:56:10,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.CompactSplit(415): Not compacting testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. because compaction request was cancelled 2023-03-14 21:56:10,353 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:10,353 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:10,355 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df5f0733ccc04ed5bfe0c04677d40708 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df5f0733ccc04ed5bfe0c04677d40708 2023-03-14 21:56:10,356 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b284c6c81a5846ecb0ed3520708e20aa to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b284c6c81a5846ecb0ed3520708e20aa 2023-03-14 21:56:10,356 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d1426d7db25b429ba1e7d85d85cf43ff to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d1426d7db25b429ba1e7d85d85cf43ff 2023-03-14 21:56:10,357 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-14 21:56:10,357 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-14 21:56:10,357 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:10,357 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:10,357 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-14 21:56:10,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-14 21:56:10,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:56:10,359 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-14 21:56:10,360 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.4710 sec 2023-03-14 21:56:10,553 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=261 (was 249) - Thread LEAK? -, OpenFileDescriptor=657 (was 623) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 254), ProcessCount=169 (was 169), AvailableMemoryMB=3167 (was 3181) 2023-03-14 21:56:10,564 INFO [Listener at localhost/45975] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=261, OpenFileDescriptor=657, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=169, AvailableMemoryMB=3167 2023-03-14 21:56:10,583 DEBUG [Listener at localhost/45975] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-14 21:56:10,619 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/.tmp/family1/483b3c82e8fe4ff89d072341e18649c9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/483b3c82e8fe4ff89d072341e18649c9 2023-03-14 21:56:10,623 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 23a9524a7f137f65decd5c4c40a09474/family1 of 23a9524a7f137f65decd5c4c40a09474 into 483b3c82e8fe4ff89d072341e18649c9(size=6.6 K), total size for store is 344.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:10,623 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:10,623 INFO [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678830969769; duration=0sec 2023-03-14 21:56:10,623 DEBUG [RS:0;jenkins-hbase4:35143-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-14 21:56:10,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10]\x0B\x9B\x8A\xDB\x17\xD1\xD4\xFA\xF4y\x8Da\xF0\x83="\x10!\xB7S\xC6J\x07\xD2\x8D\x01d\xBEI\x05*\x9D\xA6*\x10`\xF9\xC8\x82\x87\x1C\xAE#\xE3V3\xE3\xC775\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-14 21:56:10,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-14 21:56:10,641 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-14 21:56:10,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-14 21:56:10,641 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-14 21:56:10,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0 empty. 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f empty. 2023-03-14 21:56:10,644 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f empty. 2023-03-14 21:56:10,645 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:10,645 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:10,645 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:10,645 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-14 21:56:10,658 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-14 21:56:10,659 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => cc098ced3bb3c092595d9641c9f57be0, NAME => 'testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10]\x0B\x9B\x8A\xDB\x17\xD1\xD4\xFA\xF4y\x8Da\xF0\x83="\x10!\xB7S\xC6J\x07\xD2\x8D\x01d\xBEI\x05*\x9D\xA6*\x10`\xF9\xC8\x82\x87\x1C\xAE#\xE3V3\xE3\xC775\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:56:10,659 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 3449ac5f17f4db041a20b3d5e6f39a7f, NAME => 'testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10]\x0B\x9B\x8A\xDB\x17\xD1\xD4\xFA\xF4y\x8Da\xF0\x83="\x10!\xB7S\xC6J\x07\xD2\x8D\x01d\xBEI\x05*\x9D\xA6*\x10`\xF9\xC8\x82\x87\x1C\xAE#\xE3V3\xE3\xC775\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:56:10,660 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 0446aeb6869a502bba9844ed82b4594f, NAME => 'testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10]\x0B\x9B\x8A\xDB\x17\xD1\xD4\xFA\xF4y\x8Da\xF0\x83="\x10!\xB7S\xC6J\x07\xD2\x8D\x01d\xBEI\x05*\x9D\xA6*\x10`\xF9\xC8\x82\x87\x1C\xAE#\xE3V3\xE3\xC775\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/.tmp 2023-03-14 21:56:10,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_] to archive 2023-03-14 21:56:10,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:10,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/a7c8a28e6c9d42658023413dc50b8ba2 2023-03-14 21:56:10,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/f4681729029743438c6c2c5017644183 2023-03-14 21:56:10,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/32de3779e66347c8bbb96beaf2653957 2023-03-14 21:56:10,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/c8c527d38291421dbfadaecffa857118 2023-03-14 21:56:10,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/59637ecedd474f7496cec2e3eb98982f 2023-03-14 21:56:10,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/663671ee4ed044ae9bf27e51a0fa2bab 2023-03-14 21:56:10,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4780f3d553ea44bc8c52514e282681b6 2023-03-14 21:56:10,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/8c50e3fc65964a349bb36bfe62ac9b56 2023-03-14 21:56:10,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/793f5bdaa04c4934bf9a337f106037ef 2023-03-14 21:56:10,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_ to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/family1/4f05055f06ad4779a54218609f56cd1f_SeqId_44_ 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 0446aeb6869a502bba9844ed82b4594f, disabling compactions & flushes 2023-03-14 21:56:10,683 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. after waiting 0 ms 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:10,683 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:10,683 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:10,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-14 21:56:10,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing cc098ced3bb3c092595d9641c9f57be0, disabling compactions & flushes 2023-03-14 21:56:11,078 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. after waiting 0 ms 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,078 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,078 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 3449ac5f17f4db041a20b3d5e6f39a7f, disabling compactions & flushes 2023-03-14 21:56:11,083 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. after waiting 0 ms 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,083 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,083 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:11,086 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-14 21:56:11,087 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971086"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830971086"}]},"ts":"1678830971086"} 2023-03-14 21:56:11,087 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971086"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830971086"}]},"ts":"1678830971086"} 2023-03-14 21:56:11,087 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678830971086"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678830971086"}]},"ts":"1678830971086"} 2023-03-14 21:56:11,088 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-14 21:56:11,089 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-14 21:56:11,089 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830971089"}]},"ts":"1678830971089"} 2023-03-14 21:56:11,090 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-14 21:56:11,092 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc098ced3bb3c092595d9641c9f57be0, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3449ac5f17f4db041a20b3d5e6f39a7f, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0446aeb6869a502bba9844ed82b4594f, ASSIGN}] 2023-03-14 21:56:11,094 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3449ac5f17f4db041a20b3d5e6f39a7f, ASSIGN 2023-03-14 21:56:11,094 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0446aeb6869a502bba9844ed82b4594f, ASSIGN 2023-03-14 21:56:11,094 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc098ced3bb3c092595d9641c9f57be0, ASSIGN 2023-03-14 21:56:11,094 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3449ac5f17f4db041a20b3d5e6f39a7f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:56:11,094 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0446aeb6869a502bba9844ed82b4594f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:56:11,095 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc098ced3bb3c092595d9641c9f57be0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35143,1678830821197; forceNewPlan=false, retain=false 2023-03-14 21:56:11,246 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=3449ac5f17f4db041a20b3d5e6f39a7f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,246 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=0446aeb6869a502bba9844ed82b4594f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,246 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cc098ced3bb3c092595d9641c9f57be0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-14 21:56:11,247 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830971246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830971246"}]},"ts":"1678830971246"} 2023-03-14 21:56:11,247 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830971246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830971246"}]},"ts":"1678830971246"} 2023-03-14 21:56:11,247 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678830971246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678830971246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678830971246"}]},"ts":"1678830971246"} 2023-03-14 21:56:11,248 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure cc098ced3bb3c092595d9641c9f57be0, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:56:11,249 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 0446aeb6869a502bba9844ed82b4594f, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:56:11,250 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure 3449ac5f17f4db041a20b3d5e6f39a7f, server=jenkins-hbase4.apache.org,35143,1678830821197}] 2023-03-14 21:56:11,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3449ac5f17f4db041a20b3d5e6f39a7f, NAME => 'testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.', STARTKEY => '1', ENDKEY => '2'} 2023-03-14 21:56:11,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:11,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,424 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,425 DEBUG [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1 2023-03-14 21:56:11,425 DEBUG [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1 2023-03-14 21:56:11,426 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3449ac5f17f4db041a20b3d5e6f39a7f columnFamilyName family1 2023-03-14 21:56:11,426 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] regionserver.HStore(310): Store=3449ac5f17f4db041a20b3d5e6f39a7f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,427 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,428 DEBUG [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2 2023-03-14 21:56:11,428 DEBUG [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2 2023-03-14 21:56:11,428 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3449ac5f17f4db041a20b3d5e6f39a7f columnFamilyName family2 2023-03-14 21:56:11,429 INFO [StoreOpener-3449ac5f17f4db041a20b3d5e6f39a7f-1] regionserver.HStore(310): Store=3449ac5f17f4db041a20b3d5e6f39a7f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:56:11,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:11,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:56:11,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3449ac5f17f4db041a20b3d5e6f39a7f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68776990, jitterRate=0.024857014417648315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:56:11,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:11,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f., pid=122, masterSystemTime=1678830971400 2023-03-14 21:56:11,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cc098ced3bb3c092595d9641c9f57be0, NAME => 'testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.', STARTKEY => '', ENDKEY => '1'} 2023-03-14 21:56:11,437 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=3449ac5f17f4db041a20b3d5e6f39a7f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,437 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678830971437"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830971437"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830971437"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830971437"}]},"ts":"1678830971437"} 2023-03-14 21:56:11,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:11,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,439 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-03-14 21:56:11,440 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure 3449ac5f17f4db041a20b3d5e6f39a7f, server=jenkins-hbase4.apache.org,35143,1678830821197 in 188 msec 2023-03-14 21:56:11,441 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3449ac5f17f4db041a20b3d5e6f39a7f, ASSIGN in 348 msec 2023-03-14 21:56:11,458 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,460 DEBUG [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1 2023-03-14 21:56:11,460 DEBUG [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1 2023-03-14 21:56:11,460 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc098ced3bb3c092595d9641c9f57be0 columnFamilyName family1 2023-03-14 21:56:11,461 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] regionserver.HStore(310): Store=cc098ced3bb3c092595d9641c9f57be0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,461 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,462 DEBUG [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2 2023-03-14 21:56:11,462 DEBUG [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2 2023-03-14 21:56:11,462 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc098ced3bb3c092595d9641c9f57be0 columnFamilyName family2 2023-03-14 21:56:11,463 INFO [StoreOpener-cc098ced3bb3c092595d9641c9f57be0-1] regionserver.HStore(310): Store=cc098ced3bb3c092595d9641c9f57be0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:56:11,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:11,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:56:11,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cc098ced3bb3c092595d9641c9f57be0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68494001, jitterRate=0.02064014971256256}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:56:11,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:11,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0., pid=120, masterSystemTime=1678830971400 2023-03-14 21:56:11,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:11,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0446aeb6869a502bba9844ed82b4594f, NAME => 'testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.', STARTKEY => '2', ENDKEY => ''} 2023-03-14 21:56:11,470 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cc098ced3bb3c092595d9641c9f57be0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-14 21:56:11,470 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971469"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830971469"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830971469"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830971469"}]},"ts":"1678830971469"} 2023-03-14 21:56:11,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,472 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-14 21:56:11,472 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure cc098ced3bb3c092595d9641c9f57be0, server=jenkins-hbase4.apache.org,35143,1678830821197 in 223 msec 2023-03-14 21:56:11,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cc098ced3bb3c092595d9641c9f57be0, ASSIGN in 380 msec 2023-03-14 21:56:11,494 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,495 DEBUG [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1 2023-03-14 21:56:11,495 DEBUG [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1 2023-03-14 21:56:11,496 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0446aeb6869a502bba9844ed82b4594f columnFamilyName family1 2023-03-14 21:56:11,496 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] regionserver.HStore(310): Store=0446aeb6869a502bba9844ed82b4594f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,497 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,497 DEBUG [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2 2023-03-14 21:56:11,497 DEBUG [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2 2023-03-14 21:56:11,498 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0446aeb6869a502bba9844ed82b4594f columnFamilyName family2 2023-03-14 21:56:11,498 INFO [StoreOpener-0446aeb6869a502bba9844ed82b4594f-1] regionserver.HStore(310): Store=0446aeb6869a502bba9844ed82b4594f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-14 21:56:11,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-14 21:56:11,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:11,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-14 21:56:11,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0446aeb6869a502bba9844ed82b4594f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72451840, jitterRate=0.07961654663085938}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-14 21:56:11,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:11,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f., pid=121, masterSystemTime=1678830971400 2023-03-14 21:56:11,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:11,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:11,505 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=0446aeb6869a502bba9844ed82b4594f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,506 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678830971505"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678830971505"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678830971505"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678830971505"}]},"ts":"1678830971505"} 2023-03-14 21:56:11,508 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-03-14 21:56:11,508 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 0446aeb6869a502bba9844ed82b4594f, server=jenkins-hbase4.apache.org,35143,1678830821197 in 258 msec 2023-03-14 21:56:11,509 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-14 21:56:11,509 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0446aeb6869a502bba9844ed82b4594f, ASSIGN in 416 msec 2023-03-14 21:56:11,509 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-14 21:56:11,509 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678830971509"}]},"ts":"1678830971509"} 2023-03-14 21:56:11,510 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-14 21:56:11,513 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-14 21:56:11,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 874 msec 2023-03-14 21:56:11,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-14 21:56:11,748 INFO [Listener at localhost/45975] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-14 21:56:11,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:11,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:11,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:11,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:11,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-14 21:56:11,758 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-14 21:56:11,758 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:11,758 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-14 21:56:11,758 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-14 21:56:11,758 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:11,758 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:11,760 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:11,760 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:11,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:11,760 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:11,760 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:11,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:11,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:11,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:11,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:11,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-14 21:56:11,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-14 21:56:11,761 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-14 21:56:11,761 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:11,761 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-14 21:56:11,761 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:11,761 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:56:11,761 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. started... 2023-03-14 21:56:11,762 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:11,762 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:11,762 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:11,762 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0446aeb6869a502bba9844ed82b4594f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:56:11,762 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. started... 2023-03-14 21:56:11,762 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. started... 2023-03-14 21:56:11,763 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 3449ac5f17f4db041a20b3d5e6f39a7f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:56:11,764 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc098ced3bb3c092595d9641c9f57be0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-14 21:56:11,813 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303141d947ab2b79e4c31a0906bf6f126c6a8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303141d947ab2b79e4c31a0906bf6f126c6a8 2023-03-14 21:56:12,208 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303143cf87d870a0548858b0500e74be3bdd6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303143cf87d870a0548858b0500e74be3bdd6 2023-03-14 21:56:12,208 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303142df5045fc4374b558cbd709fbd08c0ab to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303142df5045fc4374b558cbd709fbd08c0ab 2023-03-14 21:56:12,223 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/89fc369472f2406e91013ecd7312643b 2023-03-14 21:56:12,223 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/5efc6c37a27f4ea0aa89c72b0d8e305e 2023-03-14 21:56:12,235 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/dd78406785bd4a50a963b399295d73e8 2023-03-14 21:56:12,665 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303143a0a1a04b26e400fad61d96512984600 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303143a0a1a04b26e400fad61d96512984600 2023-03-14 21:56:12,671 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031430592f77853f423d949d042f1ced61cd to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023031430592f77853f423d949d042f1ced61cd 2023-03-14 21:56:12,673 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314989dfc0d58674e20b841e15f16af4c69 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230314989dfc0d58674e20b841e15f16af4c69 2023-03-14 21:56:12,675 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/d3ec622bd45a40d98bef41d44e565c8c 2023-03-14 21:56:12,679 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/5efc6c37a27f4ea0aa89c72b0d8e305e as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e 2023-03-14 21:56:12,702 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e, entries=4, sequenceid=5, filesize=5.5 K 2023-03-14 21:56:12,703 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/d3ec622bd45a40d98bef41d44e565c8c as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c 2023-03-14 21:56:12,706 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:56:12,707 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cc098ced3bb3c092595d9641c9f57be0 in 944ms, sequenceid=5, compaction requested=false 2023-03-14 21:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-14 21:56:12,708 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:12,708 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:12,708 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:56:12,814 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:56:12,814 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x19affb6f to 127.0.0.1:58882 2023-03-14 21:56:12,814 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:12,815 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:56:12,815 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7614f55a to 127.0.0.1:58882 2023-03-14 21:56:12,815 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:13,070 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/3438d1af52bd46fa98316b032fb00a7f 2023-03-14 21:56:13,074 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/89fc369472f2406e91013ecd7312643b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b 2023-03-14 21:56:13,078 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/6a664dd5e6a44945944ca9ddd9753071 2023-03-14 21:56:13,081 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/dd78406785bd4a50a963b399295d73e8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8 2023-03-14 21:56:13,100 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b, entries=4, sequenceid=5, filesize=5.5 K 2023-03-14 21:56:13,100 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/3438d1af52bd46fa98316b032fb00a7f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f 2023-03-14 21:56:13,104 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8, entries=4, sequenceid=5, filesize=5.5 K 2023-03-14 21:56:13,104 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:56:13,105 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/6a664dd5e6a44945944ca9ddd9753071 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071 2023-03-14 21:56:13,105 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3449ac5f17f4db041a20b3d5e6f39a7f in 1342ms, sequenceid=5, compaction requested=false 2023-03-14 21:56:13,106 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:13,106 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:13,106 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:56:13,109 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071, entries=2, sequenceid=5, filesize=5.1 K 2023-03-14 21:56:13,109 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 0446aeb6869a502bba9844ed82b4594f in 1347ms, sequenceid=5, compaction requested=false 2023-03-14 21:56:13,110 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:13,110 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:13,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:56:13,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:56:13,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-14 21:56:13,110 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-14 21:56:13,111 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,111 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:13,112 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,112 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:13,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:13,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-14 21:56:13,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@231800c[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:13,113 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-14 21:56:13,113 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,115 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-14 21:56:13,115 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,115 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-14 21:56:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:13,116 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,116 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-14 21:56:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:13,117 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:13,117 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-14 21:56:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:13,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:13,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:13,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-14 21:56:13,119 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-14 21:56:13,119 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:13,119 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:13,119 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-14 21:56:13,119 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:13,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:13,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,121 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:13,121 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,122 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,124 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:13,124 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:13,124 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:13,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-14 21:56:13,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:13,125 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:13,125 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:13,225 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:13,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:13,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:13,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:13,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:13,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:13,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:13,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-14 21:56:13,232 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-14 21:56:13,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:13,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-14 21:56:13,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-14 21:56:13,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,232 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:13,234 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,234 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:13,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:13,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:13,235 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:13,235 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:13,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:13,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:13,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:13,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:13,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-14 21:56:13,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-14 21:56:13,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-14 21:56:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-14 21:56:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. started... 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:13,236 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0446aeb6869a502bba9844ed82b4594f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. started... 2023-03-14 21:56:13,236 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. started... 2023-03-14 21:56:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:13,236 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 3449ac5f17f4db041a20b3d5e6f39a7f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:56:13,237 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc098ced3bb3c092595d9641c9f57be0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-14 21:56:13,286 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314fbe82d5b41b347ea870a5299a86572cb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314fbe82d5b41b347ea870a5299a86572cb 2023-03-14 21:56:13,682 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314b97010e99c644345ab5649da3d3faa6f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314b97010e99c644345ab5649da3d3faa6f 2023-03-14 21:56:13,688 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303148fa9a1a3307f4df9a89a907ea2544728 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303148fa9a1a3307f4df9a89a907ea2544728 2023-03-14 21:56:13,701 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/10c3fce3ee074bcfa30b720dc1990334 2023-03-14 21:56:13,704 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/ddb73da469714486865f8a22e1409820 2023-03-14 21:56:13,709 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/9bd0393460154625a640002350e5556d 2023-03-14 21:56:13,740 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303146900adcff40b4593ae70b2c2dcafc115 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303146900adcff40b4593ae70b2c2dcafc115 2023-03-14 21:56:13,757 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303143fd35fa6eb9d447d90ed43fd51f84c69 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303143fd35fa6eb9d447d90ed43fd51f84c69 2023-03-14 21:56:13,770 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/2991dc0b8b93449a8a2c6e9b95d64a51 2023-03-14 21:56:13,773 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/ddb73da469714486865f8a22e1409820 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820 2023-03-14 21:56:13,797 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820, entries=4, sequenceid=9, filesize=5.5 K 2023-03-14 21:56:13,798 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/2991dc0b8b93449a8a2c6e9b95d64a51 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51 2023-03-14 21:56:13,801 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:56:13,802 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 0446aeb6869a502bba9844ed82b4594f in 566ms, sequenceid=9, compaction requested=false 2023-03-14 21:56:13,802 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:13,802 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:13,802 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:56:14,154 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/63b7b6b2cf5a45b6889c429217f1c0a9 2023-03-14 21:56:14,158 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/10c3fce3ee074bcfa30b720dc1990334 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334 2023-03-14 21:56:14,173 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031409d3acc7cd2540ae91e16cb60e984260 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023031409d3acc7cd2540ae91e16cb60e984260 2023-03-14 21:56:14,181 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334, entries=4, sequenceid=9, filesize=5.5 K 2023-03-14 21:56:14,182 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/63b7b6b2cf5a45b6889c429217f1c0a9 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9 2023-03-14 21:56:14,185 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:56:14,186 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3449ac5f17f4db041a20b3d5e6f39a7f in 950ms, sequenceid=9, compaction requested=false 2023-03-14 21:56:14,186 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:14,186 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:14,186 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:56:14,578 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/5bb9fb2762c544758b0d892c0b6584fe 2023-03-14 21:56:14,583 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/9bd0393460154625a640002350e5556d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d 2023-03-14 21:56:14,619 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d, entries=4, sequenceid=9, filesize=5.5 K 2023-03-14 21:56:14,620 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/5bb9fb2762c544758b0d892c0b6584fe as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe 2023-03-14 21:56:14,623 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe, entries=2, sequenceid=9, filesize=5.1 K 2023-03-14 21:56:14,624 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cc098ced3bb3c092595d9641c9f57be0 in 1387ms, sequenceid=9, compaction requested=false 2023-03-14 21:56:14,624 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:14,624 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:56:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:56:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-14 21:56:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-14 21:56:14,626 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:14,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:14,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:14,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-14 21:56:14,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4caebd2d[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:14,628 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-14 21:56:14,628 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,629 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,629 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-14 21:56:14,629 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-14 21:56:14,629 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:14,629 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,629 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:14,630 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,630 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-14 21:56:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:14,630 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-14 21:56:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:14,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:14,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-14 21:56:14,632 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-14 21:56:14,632 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:14,632 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:14,632 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-14 21:56:14,632 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:14,634 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,634 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,634 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:14,635 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:14,636 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:14,638 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:14,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-14 21:56:14,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:14,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:14,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,638 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:14,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:14,639 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,738 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:14,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:14,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:14,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:14,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:14,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:14,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:14,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-14 21:56:14,744 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-14 21:56:14,744 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:14,744 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-14 21:56:14,744 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-14 21:56:14,745 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,745 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:14,746 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:14,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:14,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:14,746 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:14,747 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:14,747 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:14,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:14,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:14,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:14,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-14 21:56:14,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-14 21:56:14,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-14 21:56:14,748 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-14 21:56:14,748 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:14,748 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-14 21:56:14,751 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:14,751 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:14,751 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. started... 2023-03-14 21:56:14,752 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. started... 2023-03-14 21:56:14,751 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:14,751 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:56:14,752 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc098ced3bb3c092595d9641c9f57be0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:56:14,752 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:14,752 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0446aeb6869a502bba9844ed82b4594f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:56:14,752 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. started... 2023-03-14 21:56:14,752 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 3449ac5f17f4db041a20b3d5e6f39a7f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-14 21:56:14,814 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314416b8adcc9664aa8b91f4d783018885b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314416b8adcc9664aa8b91f4d783018885b 2023-03-14 21:56:14,818 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303144ea09e22ac824734b03c602630deac94 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303144ea09e22ac824734b03c602630deac94 2023-03-14 21:56:15,211 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303145d59b0f6965d4ad1bab65fae03d184e5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145d59b0f6965d4ad1bab65fae03d184e5 2023-03-14 21:56:15,233 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/24ccf536760744aeabc4fefd6c107ee8 2023-03-14 21:56:15,239 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/c93dfb445a7648c7bf291a6d64546737 2023-03-14 21:56:15,269 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031422e2c2b366394e7f80234733084e337d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023031422e2c2b366394e7f80234733084e337d 2023-03-14 21:56:15,273 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/3f039f42f0d84b7b99e665dff139b0e3 2023-03-14 21:56:15,277 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/24ccf536760744aeabc4fefd6c107ee8 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8 2023-03-14 21:56:15,299 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8, entries=4, sequenceid=13, filesize=5.5 K 2023-03-14 21:56:15,300 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/3f039f42f0d84b7b99e665dff139b0e3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3 2023-03-14 21:56:15,303 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:56:15,304 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3449ac5f17f4db041a20b3d5e6f39a7f in 552ms, sequenceid=13, compaction requested=true 2023-03-14 21:56:15,304 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:15,304 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:56:15,626 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/2ba1159b01e445208312a0c8a86c6b32 2023-03-14 21:56:15,676 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314e092d31700c940a78d4eb74da56d6f0e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230314e092d31700c940a78d4eb74da56d6f0e 2023-03-14 21:56:16,062 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314886982228e8d4ad0b8ea8633793762b2 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230314886982228e8d4ad0b8ea8633793762b2 2023-03-14 21:56:16,080 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/794fa04f2d0445ad986095bf9e1df317 2023-03-14 21:56:16,084 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/c93dfb445a7648c7bf291a6d64546737 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737 2023-03-14 21:56:16,118 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737, entries=4, sequenceid=13, filesize=5.5 K 2023-03-14 21:56:16,118 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/794fa04f2d0445ad986095bf9e1df317 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317 2023-03-14 21:56:16,122 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:56:16,122 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cc098ced3bb3c092595d9641c9f57be0 in 1370ms, sequenceid=13, compaction requested=true 2023-03-14 21:56:16,122 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:16,122 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:16,122 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:56:16,466 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/9b6c9e971ab0470483b2bb7ea5b50c26 2023-03-14 21:56:16,470 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/2ba1159b01e445208312a0c8a86c6b32 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32 2023-03-14 21:56:16,493 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32, entries=4, sequenceid=13, filesize=5.5 K 2023-03-14 21:56:16,493 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/9b6c9e971ab0470483b2bb7ea5b50c26 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26 2023-03-14 21:56:16,496 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26, entries=2, sequenceid=13, filesize=5.1 K 2023-03-14 21:56:16,497 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 0446aeb6869a502bba9844ed82b4594f in 1745ms, sequenceid=13, compaction requested=true 2023-03-14 21:56:16,497 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:16,497 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:16,497 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:56:16,497 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:56:16,497 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,497 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-14 21:56:16,497 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-14 21:56:16,499 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:16,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,499 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:16,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-14 21:56:16,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@424bbb5f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:16,501 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-14 21:56:16,501 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,503 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,503 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-14 21:56:16,503 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,503 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:16,503 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-14 21:56:16,503 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:16,505 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,505 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-14 21:56:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,505 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:16,505 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-14 21:56:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:16,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-14 21:56:16,507 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-14 21:56:16,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:16,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:16,507 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-14 21:56:16,507 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:16,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,508 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:16,509 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:16,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,510 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:16,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:16,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,512 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:16,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-14 21:56:16,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:16,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-14 21:56:16,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,512 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:16,513 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:16,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:16,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:16,613 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:16,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:16,614 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:16,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:16,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-14 21:56:16,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:16,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:16,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-14 21:56:16,619 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-14 21:56:16,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:16,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-14 21:56:16,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-14 21:56:16,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,619 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:16,623 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:16,623 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:16,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:16,624 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:16,624 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:16,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:16,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:16,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-14 21:56:16,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-14 21:56:16,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-14 21:56:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-14 21:56:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-14 21:56:16,625 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:16,625 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:56:16,626 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:16,625 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. started... 2023-03-14 21:56:16,625 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:16,626 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. started... 2023-03-14 21:56:16,625 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. started... 2023-03-14 21:56:16,626 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc098ced3bb3c092595d9641c9f57be0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:56:16,626 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 3449ac5f17f4db041a20b3d5e6f39a7f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:56:16,626 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0446aeb6869a502bba9844ed82b4594f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-14 21:56:16,680 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303145afaec6919964e8ca38fcf58e2d88b4c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145afaec6919964e8ca38fcf58e2d88b4c 2023-03-14 21:56:16,685 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031411a2e94d78e84b3d877ff60b943870ee to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031411a2e94d78e84b3d877ff60b943870ee 2023-03-14 21:56:16,904 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-14 21:56:17,073 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230314c2af7042208149f6932c0cd0ea62c9b8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314c2af7042208149f6932c0cd0ea62c9b8 2023-03-14 21:56:17,097 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3 2023-03-14 21:56:17,101 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/68ac88399b474c8e858244ffdfab3e86 2023-03-14 21:56:17,487 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/451c8197a6de4c7f9b0ad02485b27312 2023-03-14 21:56:17,531 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303146b167887870047aa82502315fa601cee to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303146b167887870047aa82502315fa601cee 2023-03-14 21:56:17,532 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314ec7645fe24554d5e88639387c9870a3a to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230314ec7645fe24554d5e88639387c9870a3a 2023-03-14 21:56:17,920 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031471a07d75ef014fbda9dd358a3da2cdbb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023031471a07d75ef014fbda9dd358a3da2cdbb 2023-03-14 21:56:17,937 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/9464d86ca9e44ab4bcae1da14ccecfd1 2023-03-14 21:56:17,937 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/094b925e7148437893d3e7ffa170cab0 2023-03-14 21:56:17,940 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3 2023-03-14 21:56:17,940 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/68ac88399b474c8e858244ffdfab3e86 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86 2023-03-14 21:56:17,965 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86, entries=4, sequenceid=17, filesize=5.5 K 2023-03-14 21:56:17,965 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/9464d86ca9e44ab4bcae1da14ccecfd1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1 2023-03-14 21:56:17,969 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:56:17,970 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cc098ced3bb3c092595d9641c9f57be0 in 1344ms, sequenceid=17, compaction requested=true 2023-03-14 21:56:17,970 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:17,970 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:17,970 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:56:17,972 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3, entries=4, sequenceid=17, filesize=5.5 K 2023-03-14 21:56:17,973 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/094b925e7148437893d3e7ffa170cab0 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0 2023-03-14 21:56:17,976 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:56:17,977 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 0446aeb6869a502bba9844ed82b4594f in 1351ms, sequenceid=17, compaction requested=true 2023-03-14 21:56:17,977 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:17,977 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:17,977 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:56:18,325 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/64d845698ebb44888d973a8060dd2470 2023-03-14 21:56:18,329 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/451c8197a6de4c7f9b0ad02485b27312 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312 2023-03-14 21:56:18,363 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312, entries=4, sequenceid=17, filesize=5.5 K 2023-03-14 21:56:18,364 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/64d845698ebb44888d973a8060dd2470 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470 2023-03-14 21:56:18,367 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470, entries=2, sequenceid=17, filesize=5.1 K 2023-03-14 21:56:18,368 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3449ac5f17f4db041a20b3d5e6f39a7f in 1742ms, sequenceid=17, compaction requested=true 2023-03-14 21:56:18,368 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:18,368 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:18,368 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:56:18,368 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:56:18,368 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,369 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-14 21:56:18,369 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-14 21:56:18,370 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,370 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:18,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:18,371 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,371 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:18,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:18,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:18,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:18,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-14 21:56:18,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@406bc59e[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:18,372 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-14 21:56:18,372 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,373 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,374 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-14 21:56:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-14 21:56:18,374 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-14 21:56:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:18,376 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-14 21:56:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:18,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:18,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-14 21:56:18,378 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-14 21:56:18,378 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:18,378 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:18,378 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-14 21:56:18,378 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:18,380 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:18,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:18,380 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,381 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:18,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:18,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:18,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:18,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-14 21:56:18,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:18,383 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:18,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:18,383 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:18,384 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,484 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:18,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:18,491 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:18,497 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:18,504 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:18,505 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:18,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-14 21:56:18,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-14 21:56:18,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-14 21:56:18,589 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-14 21:56:18,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:18,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-14 21:56:18,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-14 21:56:18,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,589 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:18,590 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:18,590 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:18,590 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:18,591 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-14 21:56:18,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-14 21:56:18,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-14 21:56:18,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-14 21:56:18,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-14 21:56:18,591 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. started... 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. started... 2023-03-14 21:56:18,592 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-14 21:56:18,592 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0446aeb6869a502bba9844ed82b4594f 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:56:18,592 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 3449ac5f17f4db041a20b3d5e6f39a7f 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:56:18,592 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:18,592 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. started... 2023-03-14 21:56:18,592 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc098ced3bb3c092595d9641c9f57be0 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-14 21:56:18,632 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/9405356dd97a4bd7b771166cf8dcd0fc 2023-03-14 21:56:18,641 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/677f3d65a3554275bd99942196141044 2023-03-14 21:56:18,657 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9405356dd97a4bd7b771166cf8dcd0fc 2023-03-14 21:56:18,666 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 677f3d65a3554275bd99942196141044 2023-03-14 21:56:19,428 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/4e3bdcc68ce24d349903da19fa206e37 2023-03-14 21:56:19,451 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e3bdcc68ce24d349903da19fa206e37 2023-03-14 21:56:19,471 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/5a910fffe8074770a8fb31b2f5fdd71b 2023-03-14 21:56:19,474 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a910fffe8074770a8fb31b2f5fdd71b 2023-03-14 21:56:19,474 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/9405356dd97a4bd7b771166cf8dcd0fc as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc 2023-03-14 21:56:19,475 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/093881dcb97c4278852c8c58d4b6bd69 2023-03-14 21:56:19,478 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 093881dcb97c4278852c8c58d4b6bd69 2023-03-14 21:56:19,478 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/677f3d65a3554275bd99942196141044 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044 2023-03-14 21:56:19,505 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9405356dd97a4bd7b771166cf8dcd0fc 2023-03-14 21:56:19,505 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc, entries=3, sequenceid=23, filesize=5.2 K 2023-03-14 21:56:19,506 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/5a910fffe8074770a8fb31b2f5fdd71b as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b 2023-03-14 21:56:19,508 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 677f3d65a3554275bd99942196141044 2023-03-14 21:56:19,508 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044, entries=3, sequenceid=23, filesize=5.2 K 2023-03-14 21:56:19,509 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/093881dcb97c4278852c8c58d4b6bd69 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69 2023-03-14 21:56:19,510 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a910fffe8074770a8fb31b2f5fdd71b 2023-03-14 21:56:19,510 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:56:19,511 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3449ac5f17f4db041a20b3d5e6f39a7f in 919ms, sequenceid=23, compaction requested=true 2023-03-14 21:56:19,511 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:19,511 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:19,511 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-14 21:56:19,513 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 093881dcb97c4278852c8c58d4b6bd69 2023-03-14 21:56:19,513 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:56:19,513 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 0446aeb6869a502bba9844ed82b4594f in 921ms, sequenceid=23, compaction requested=true 2023-03-14 21:56:19,514 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:19,514 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:19,514 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-14 21:56:20,262 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/bcfc0e1165ef46d1805d9defcedb5f64 2023-03-14 21:56:20,265 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcfc0e1165ef46d1805d9defcedb5f64 2023-03-14 21:56:20,266 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/4e3bdcc68ce24d349903da19fa206e37 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37 2023-03-14 21:56:20,288 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e3bdcc68ce24d349903da19fa206e37 2023-03-14 21:56:20,288 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37, entries=3, sequenceid=23, filesize=5.2 K 2023-03-14 21:56:20,289 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/bcfc0e1165ef46d1805d9defcedb5f64 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64 2023-03-14 21:56:20,292 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcfc0e1165ef46d1805d9defcedb5f64 2023-03-14 21:56:20,292 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64, entries=1, sequenceid=23, filesize=5.0 K 2023-03-14 21:56:20,294 INFO [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cc098ced3bb3c092595d9641c9f57be0 in 1701ms, sequenceid=23, compaction requested=true 2023-03-14 21:56:20,294 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:20,294 DEBUG [rs(jenkins-hbase4.apache.org,35143,1678830821197)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:20,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-14 21:56:20,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-14 21:56:20,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-14 21:56:20,294 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-14 21:56:20,295 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,295 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:20,296 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,296 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-14 21:56:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:20,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35143,1678830821197' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-14 21:56:20,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ed41c7f[Count = 0] remaining members to acquire global barrier 2023-03-14 21:56:20,297 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-14 21:56:20,297 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,298 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-14 21:56:20,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-14 21:56:20,298 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,35143,1678830821197' in zk 2023-03-14 21:56:20,298 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,298 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-14 21:56:20,299 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,299 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-14 21:56:20,300 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:20,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:20,300 DEBUG [member: 'jenkins-hbase4.apache.org,35143,1678830821197' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:20,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,35143,1678830821197': 2023-03-14 21:56:20,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35143,1678830821197' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-14 21:56:20,301 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-14 21:56:20,302 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-14 21:56:20,302 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-14 21:56:20,302 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-14 21:56:20,302 INFO [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-14 21:56:20,309 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-14 21:56:20,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:20,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:20,309 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-14 21:56:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,310 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-14 21:56:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-14 21:56:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:20,312 DEBUG [(jenkins-hbase4.apache.org,40679,1678830820017)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-14 21:56:20,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:20,312 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,313 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-14 21:56:20,313 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-14 21:56:20,313 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-14 21:56:20,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-14 21:56:20,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-14 21:56:20,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-14 21:56:20,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-14 21:56:20,321 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-14 21:56:20,321 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-14 21:56:20,421 DEBUG [Listener at localhost/45975] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-14 21:56:20,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40679] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-14 21:56:20,422 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:20,422 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 0446aeb6869a502bba9844ed82b4594f/family1 is initiating major compaction (all files) 2023-03-14 21:56:20,422 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:20,422 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:20,423 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 0446aeb6869a502bba9844ed82b4594f/family1 in testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:20,423 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp, totalSize=27.1 K 2023-03-14 21:56:20,423 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting dd78406785bd4a50a963b399295d73e8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971754 2023-03-14 21:56:20,424 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting ddb73da469714486865f8a22e1409820, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973228 2023-03-14 21:56:20,424 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 2ba1159b01e445208312a0c8a86c6b32, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974741 2023-03-14 21:56:20,424 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting e75fabd08edf4083b3ce9e0bd5e9e6a3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976615 2023-03-14 21:56:20,425 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 677f3d65a3554275bd99942196141044, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:20,433 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 0446aeb6869a502bba9844ed82b4594f#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:20,870 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del 2023-03-14 21:56:20,871 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del 2023-03-14 21:56:21,726 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa61247abef446e3aaa8d0e0aff54f53 2023-03-14 21:56:21,726 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family1/aa61247abef446e3aaa8d0e0aff54f53 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/aa61247abef446e3aaa8d0e0aff54f53 2023-03-14 21:56:21,756 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa61247abef446e3aaa8d0e0aff54f53 2023-03-14 21:56:21,757 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0446aeb6869a502bba9844ed82b4594f/family1 of 0446aeb6869a502bba9844ed82b4594f into aa61247abef446e3aaa8d0e0aff54f53(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-14 21:56:21,757 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:21,757 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:21,757 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 0446aeb6869a502bba9844ed82b4594f/family2 is initiating major compaction (all files) 2023-03-14 21:56:21,757 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:21,757 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:21,757 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 0446aeb6869a502bba9844ed82b4594f/family2 in testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:21,757 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp, totalSize=25.4 K 2023-03-14 21:56:21,758 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 6a664dd5e6a44945944ca9ddd9753071, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971754 2023-03-14 21:56:21,758 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 2991dc0b8b93449a8a2c6e9b95d64a51, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973228 2023-03-14 21:56:21,759 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9b6c9e971ab0470483b2bb7ea5b50c26, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974741 2023-03-14 21:56:21,759 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 094b925e7148437893d3e7ffa170cab0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976615 2023-03-14 21:56:21,759 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 093881dcb97c4278852c8c58d4b6bd69, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:21,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044] to archive 2023-03-14 21:56:21,766 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:21,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/dd78406785bd4a50a963b399295d73e8 2023-03-14 21:56:21,768 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 0446aeb6869a502bba9844ed82b4594f#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:21,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/ddb73da469714486865f8a22e1409820 2023-03-14 21:56:21,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/2ba1159b01e445208312a0c8a86c6b32 2023-03-14 21:56:21,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/e75fabd08edf4083b3ce9e0bd5e9e6a3 2023-03-14 21:56:21,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/677f3d65a3554275bd99942196141044 2023-03-14 21:56:21,775 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023031486f92b944a064402bfd9a48d2e1fc48b_del 2023-03-14 21:56:21,776 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023031486f92b944a064402bfd9a48d2e1fc48b_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023031486f92b944a064402bfd9a48d2e1fc48b_del 2023-03-14 21:56:22,589 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d8cb785d8744452e8415eace6ad90f90 2023-03-14 21:56:22,590 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/.tmp/family2/d8cb785d8744452e8415eace6ad90f90 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/d8cb785d8744452e8415eace6ad90f90 2023-03-14 21:56:22,592 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d8cb785d8744452e8415eace6ad90f90 2023-03-14 21:56:22,593 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0446aeb6869a502bba9844ed82b4594f/family2 of 0446aeb6869a502bba9844ed82b4594f into d8cb785d8744452e8415eace6ad90f90(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:22,593 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:22,594 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:22,594 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 3449ac5f17f4db041a20b3d5e6f39a7f/family1 is initiating major compaction (all files) 2023-03-14 21:56:22,594 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:22,594 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:22,594 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 3449ac5f17f4db041a20b3d5e6f39a7f/family1 in testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:22,594 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp, totalSize=27.1 K 2023-03-14 21:56:22,595 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 89fc369472f2406e91013ecd7312643b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971754 2023-03-14 21:56:22,595 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 10c3fce3ee074bcfa30b720dc1990334, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973227 2023-03-14 21:56:22,596 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 24ccf536760744aeabc4fefd6c107ee8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974740 2023-03-14 21:56:22,596 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 451c8197a6de4c7f9b0ad02485b27312, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976615 2023-03-14 21:56:22,596 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9405356dd97a4bd7b771166cf8dcd0fc, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:22,604 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 3449ac5f17f4db041a20b3d5e6f39a7f#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:22,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69] to archive 2023-03-14 21:56:22,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:22,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/6a664dd5e6a44945944ca9ddd9753071 2023-03-14 21:56:22,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/2991dc0b8b93449a8a2c6e9b95d64a51 2023-03-14 21:56:22,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/9b6c9e971ab0470483b2bb7ea5b50c26 2023-03-14 21:56:22,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/094b925e7148437893d3e7ffa170cab0 2023-03-14 21:56:22,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family2/093881dcb97c4278852c8c58d4b6bd69 2023-03-14 21:56:23,048 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del 2023-03-14 21:56:23,048 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del 2023-03-14 21:56:23,501 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db4cce77a563481696c7d20f675c084f 2023-03-14 21:56:23,502 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family1/db4cce77a563481696c7d20f675c084f as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/db4cce77a563481696c7d20f675c084f 2023-03-14 21:56:23,527 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db4cce77a563481696c7d20f675c084f 2023-03-14 21:56:23,528 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3449ac5f17f4db041a20b3d5e6f39a7f/family1 of 3449ac5f17f4db041a20b3d5e6f39a7f into db4cce77a563481696c7d20f675c084f(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:23,529 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:23,529 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:23,529 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): 3449ac5f17f4db041a20b3d5e6f39a7f/family2 is initiating major compaction (all files) 2023-03-14 21:56:23,529 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:23,529 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:23,529 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of 3449ac5f17f4db041a20b3d5e6f39a7f/family2 in testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:23,530 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp, totalSize=25.4 K 2023-03-14 21:56:23,531 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 3438d1af52bd46fa98316b032fb00a7f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971754 2023-03-14 21:56:23,531 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 63b7b6b2cf5a45b6889c429217f1c0a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973227 2023-03-14 21:56:23,532 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 3f039f42f0d84b7b99e665dff139b0e3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974740 2023-03-14 21:56:23,532 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 64d845698ebb44888d973a8060dd2470, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976615 2023-03-14 21:56:23,533 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5a910fffe8074770a8fb31b2f5fdd71b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:23,552 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): 3449ac5f17f4db041a20b3d5e6f39a7f#family2#compaction#200 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:23,559 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023031489ceb98dd4eb49c2aa4541fb86937e84_del 2023-03-14 21:56:23,559 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023031489ceb98dd4eb49c2aa4541fb86937e84_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023031489ceb98dd4eb49c2aa4541fb86937e84_del 2023-03-14 21:56:23,567 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc] to archive 2023-03-14 21:56:23,568 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:23,569 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/89fc369472f2406e91013ecd7312643b 2023-03-14 21:56:23,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/10c3fce3ee074bcfa30b720dc1990334 2023-03-14 21:56:23,571 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/24ccf536760744aeabc4fefd6c107ee8 2023-03-14 21:56:23,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/451c8197a6de4c7f9b0ad02485b27312 2023-03-14 21:56:23,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/9405356dd97a4bd7b771166cf8dcd0fc 2023-03-14 21:56:23,970 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69028d53143e45e8bad7bce3f7618af4 2023-03-14 21:56:23,971 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/.tmp/family2/69028d53143e45e8bad7bce3f7618af4 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/69028d53143e45e8bad7bce3f7618af4 2023-03-14 21:56:23,975 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69028d53143e45e8bad7bce3f7618af4 2023-03-14 21:56:23,976 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3449ac5f17f4db041a20b3d5e6f39a7f/family2 of 3449ac5f17f4db041a20b3d5e6f39a7f into 69028d53143e45e8bad7bce3f7618af4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:23,977 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:23,977 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:23,977 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): cc098ced3bb3c092595d9641c9f57be0/family1 is initiating major compaction (all files) 2023-03-14 21:56:23,977 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:23,977 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:23,977 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of cc098ced3bb3c092595d9641c9f57be0/family1 in testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:23,977 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp, totalSize=27.1 K 2023-03-14 21:56:23,978 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5efc6c37a27f4ea0aa89c72b0d8e305e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971753 2023-03-14 21:56:23,978 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9bd0393460154625a640002350e5556d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973227 2023-03-14 21:56:23,979 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting c93dfb445a7648c7bf291a6d64546737, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974740 2023-03-14 21:56:23,979 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 68ac88399b474c8e858244ffdfab3e86, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976614 2023-03-14 21:56:23,979 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 4e3bdcc68ce24d349903da19fa206e37, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:23,989 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): cc098ced3bb3c092595d9641c9f57be0#family1#compaction#201 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:24,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b] to archive 2023-03-14 21:56:24,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:24,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3438d1af52bd46fa98316b032fb00a7f 2023-03-14 21:56:24,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/63b7b6b2cf5a45b6889c429217f1c0a9 2023-03-14 21:56:24,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/3f039f42f0d84b7b99e665dff139b0e3 2023-03-14 21:56:24,078 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/64d845698ebb44888d973a8060dd2470 2023-03-14 21:56:24,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family2/5a910fffe8074770a8fb31b2f5fdd71b 2023-03-14 21:56:24,431 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del 2023-03-14 21:56:24,431 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del 2023-03-14 21:56:24,898 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7329b7a7a11e49399650096587b9ae86 2023-03-14 21:56:24,898 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family1/7329b7a7a11e49399650096587b9ae86 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/7329b7a7a11e49399650096587b9ae86 2023-03-14 21:56:24,920 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7329b7a7a11e49399650096587b9ae86 2023-03-14 21:56:24,921 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc098ced3bb3c092595d9641c9f57be0/family1 of cc098ced3bb3c092595d9641c9f57be0 into 7329b7a7a11e49399650096587b9ae86(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:24,921 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:24,921 DEBUG [Listener at localhost/45975] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-14 21:56:24,922 DEBUG [Listener at localhost/45975] regionserver.HStore(1912): cc098ced3bb3c092595d9641c9f57be0/family2 is initiating major compaction (all files) 2023-03-14 21:56:24,922 INFO [Listener at localhost/45975] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-14 21:56:24,922 INFO [Listener at localhost/45975] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-14 21:56:24,922 INFO [Listener at localhost/45975] regionserver.HRegion(2259): Starting compaction of cc098ced3bb3c092595d9641c9f57be0/family2 in testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:24,922 INFO [Listener at localhost/45975] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64] into tmpdir=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp, totalSize=25.4 K 2023-03-14 21:56:24,922 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting d3ec622bd45a40d98bef41d44e565c8c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678830971753 2023-03-14 21:56:24,923 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 5bb9fb2762c544758b0d892c0b6584fe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678830973227 2023-03-14 21:56:24,923 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 794fa04f2d0445ad986095bf9e1df317, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678830974740 2023-03-14 21:56:24,923 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting 9464d86ca9e44ab4bcae1da14ccecfd1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678830976614 2023-03-14 21:56:24,923 DEBUG [Listener at localhost/45975] compactions.Compactor(207): Compacting bcfc0e1165ef46d1805d9defcedb5f64, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-14 21:56:24,933 INFO [Listener at localhost/45975] throttle.PressureAwareThroughputController(145): cc098ced3bb3c092595d9641c9f57be0#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-14 21:56:24,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37] to archive 2023-03-14 21:56:24,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:24,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/5efc6c37a27f4ea0aa89c72b0d8e305e 2023-03-14 21:56:24,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/9bd0393460154625a640002350e5556d 2023-03-14 21:56:24,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/c93dfb445a7648c7bf291a6d64546737 2023-03-14 21:56:24,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/68ac88399b474c8e858244ffdfab3e86 2023-03-14 21:56:24,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/4e3bdcc68ce24d349903da19fa206e37 2023-03-14 21:56:25,340 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230314fbdfde9309d643ffbfeaa534e82cf0b4_del 2023-03-14 21:56:25,340 INFO [Listener at localhost/45975] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230314fbdfde9309d643ffbfeaa534e82cf0b4_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230314fbdfde9309d643ffbfeaa534e82cf0b4_del 2023-03-14 21:56:25,754 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12a3219ed33146dbb74bf458922700ac 2023-03-14 21:56:25,754 DEBUG [Listener at localhost/45975] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/.tmp/family2/12a3219ed33146dbb74bf458922700ac as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/12a3219ed33146dbb74bf458922700ac 2023-03-14 21:56:25,758 INFO [Listener at localhost/45975] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12a3219ed33146dbb74bf458922700ac 2023-03-14 21:56:25,759 INFO [Listener at localhost/45975] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc098ced3bb3c092595d9641c9f57be0/family2 of cc098ced3bb3c092595d9641c9f57be0 into 12a3219ed33146dbb74bf458922700ac(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-14 21:56:25,760 DEBUG [Listener at localhost/45975] regionserver.HRegion(2289): Compaction status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:25,763 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:25,765 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:25,768 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:25,769 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:25,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64] to archive 2023-03-14 21:56:25,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:25,774 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40679] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10]\x0B\x9B\x8A\xDB\x17\xD1\xD4\xFA\xF4y\x8Da\xF0\x83="\x10!\xB7S\xC6J\x07\xD2\x8D\x01d\xBEI\x05*\x9D\xA6*\x10`\xF9\xC8\x82\x87\x1C\xAE#\xE3V3\xE3\xC775\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-14 21:56:25,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/d3ec622bd45a40d98bef41d44e565c8c 2023-03-14 21:56:25,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/5bb9fb2762c544758b0d892c0b6584fe 2023-03-14 21:56:25,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/794fa04f2d0445ad986095bf9e1df317 2023-03-14 21:56:25,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/9464d86ca9e44ab4bcae1da14ccecfd1 2023-03-14 21:56:25,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family2/bcfc0e1165ef46d1805d9defcedb5f64 2023-03-14 21:56:25,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-14 21:56:25,803 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-14 21:56:25,804 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:56:25,804 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678831585804 2023-03-14 21:56:25,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-14 21:56:26,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-14 21:56:26,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del 2023-03-14 21:56:26,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del 2023-03-14 21:56:26,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del 2023-03-14 21:56:26,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:26,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] zookeeper.ReadOnlyZKClient(139): Connect 0x56f88916 to 127.0.0.1:58882 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-14 21:56:26,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c2c0be7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-14 21:56:26,308 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:27,298 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023031403c0ea3e391f44a5b0a697c6944a1a9b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031403c0ea3e391f44a5b0a697c6944a1a9b 2023-03-14 21:56:27,299 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:27,316 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-14 21:56:27,318 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44456, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-14 21:56:27,327 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-14 21:56:27,329 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41584, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-14 21:56:27,356 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230314/family1/3f53a71c70c048c9bfca18ff01d90edf first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-14 21:56:27,357 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230314/family1/3f53a71c70c048c9bfca18ff01d90edf}] 2023-03-14 21:56:27,362 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230314/family1/3f53a71c70c048c9bfca18ff01d90edf for inclusion in cc098ced3bb3c092595d9641c9f57be0/family1 2023-03-14 21:56:27,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-14 21:56:27,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first= last=1 2023-03-14 21:56:27,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:27,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230314/family1/3f53a71c70c048c9bfca18ff01d90edf to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__hhh2ehpm8cc21m2mqdu014e41h0b4j01ts9qgjuf7p2nqar1p7j1lmutngnh1t6s/family1/3f53a71c70c048c9bfca18ff01d90edf 2023-03-14 21:56:27,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__hhh2ehpm8cc21m2mqdu014e41h0b4j01ts9qgjuf7p2nqar1p7j1lmutngnh1t6s/family1/3f53a71c70c048c9bfca18ff01d90edf as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/3bd2e728cf884294b4b1c215c0ae2c3c_SeqId_28_ 2023-03-14 21:56:27,386 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__hhh2ehpm8cc21m2mqdu014e41h0b4j01ts9qgjuf7p2nqar1p7j1lmutngnh1t6s/family1/3f53a71c70c048c9bfca18ff01d90edf into cc098ced3bb3c092595d9641c9f57be0/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/3bd2e728cf884294b4b1c215c0ae2c3c_SeqId_28_ - updating store file list. 2023-03-14 21:56:27,408 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/3bd2e728cf884294b4b1c215c0ae2c3c_SeqId_28_ into cc098ced3bb3c092595d9641c9f57be0/family1 2023-03-14 21:56:27,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__hhh2ehpm8cc21m2mqdu014e41h0b4j01ts9qgjuf7p2nqar1p7j1lmutngnh1t6s/family1/3f53a71c70c048c9bfca18ff01d90edf into cc098ced3bb3c092595d9641c9f57be0/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/family1/3bd2e728cf884294b4b1c215c0ae2c3c_SeqId_28_) 2023-03-14 21:56:27,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__hhh2ehpm8cc21m2mqdu014e41h0b4j01ts9qgjuf7p2nqar1p7j1lmutngnh1t6s/family1/3f53a71c70c048c9bfca18ff01d90edf 2023-03-14 21:56:27,411 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:27,411 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:27,412 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031411a2e94d78e84b3d877ff60b943870ee to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023031411a2e94d78e84b3d877ff60b943870ee 2023-03-14 21:56:27,413 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303143cf87d870a0548858b0500e74be3bdd6 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303143cf87d870a0548858b0500e74be3bdd6 2023-03-14 21:56:27,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303144ea09e22ac824734b03c602630deac94 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303144ea09e22ac824734b03c602630deac94 2023-03-14 21:56:27,415 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303148fa9a1a3307f4df9a89a907ea2544728 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303148fa9a1a3307f4df9a89a907ea2544728 2023-03-14 21:56:27,415 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-14 21:56:27,415 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230314' 2023-03-14 21:56:27,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:27,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:27,979 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303143a617e0173fc450aaa7e50836eb72ce1 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303143a617e0173fc450aaa7e50836eb72ce1 2023-03-14 21:56:27,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:28,037 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230314/family1/654b5e1c79d345beb84f5dbd4f36b06d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-14 21:56:28,038 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230314/family1/654b5e1c79d345beb84f5dbd4f36b06d}] 2023-03-14 21:56:28,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230314/family1/654b5e1c79d345beb84f5dbd4f36b06d for inclusion in 3449ac5f17f4db041a20b3d5e6f39a7f/family1 2023-03-14 21:56:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-14 21:56:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-14 21:56:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230314/family1/654b5e1c79d345beb84f5dbd4f36b06d to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__f06q9lod2ta300b88fu8dqm840vhgmflvo0rraob2kbb0sn8c280c4fb4ascruv3/family1/654b5e1c79d345beb84f5dbd4f36b06d 2023-03-14 21:56:28,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__f06q9lod2ta300b88fu8dqm840vhgmflvo0rraob2kbb0sn8c280c4fb4ascruv3/family1/654b5e1c79d345beb84f5dbd4f36b06d as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/f29d5503909a40419c9fcfd20d4af8da_SeqId_28_ 2023-03-14 21:56:28,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__f06q9lod2ta300b88fu8dqm840vhgmflvo0rraob2kbb0sn8c280c4fb4ascruv3/family1/654b5e1c79d345beb84f5dbd4f36b06d into 3449ac5f17f4db041a20b3d5e6f39a7f/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/f29d5503909a40419c9fcfd20d4af8da_SeqId_28_ - updating store file list. 2023-03-14 21:56:28,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/f29d5503909a40419c9fcfd20d4af8da_SeqId_28_ into 3449ac5f17f4db041a20b3d5e6f39a7f/family1 2023-03-14 21:56:28,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__f06q9lod2ta300b88fu8dqm840vhgmflvo0rraob2kbb0sn8c280c4fb4ascruv3/family1/654b5e1c79d345beb84f5dbd4f36b06d into 3449ac5f17f4db041a20b3d5e6f39a7f/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/family1/f29d5503909a40419c9fcfd20d4af8da_SeqId_28_) 2023-03-14 21:56:28,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__f06q9lod2ta300b88fu8dqm840vhgmflvo0rraob2kbb0sn8c280c4fb4ascruv3/family1/654b5e1c79d345beb84f5dbd4f36b06d 2023-03-14 21:56:28,090 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:28,091 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:28,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303142df5045fc4374b558cbd709fbd08c0ab to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303142df5045fc4374b558cbd709fbd08c0ab 2023-03-14 21:56:28,093 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314416b8adcc9664aa8b91f4d783018885b to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314416b8adcc9664aa8b91f4d783018885b 2023-03-14 21:56:28,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314b97010e99c644345ab5649da3d3faa6f to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314b97010e99c644345ab5649da3d3faa6f 2023-03-14 21:56:28,095 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314c2af7042208149f6932c0cd0ea62c9b8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230314c2af7042208149f6932c0cd0ea62c9b8 2023-03-14 21:56:28,095 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-14 21:56:28,095 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230314' 2023-03-14 21:56:28,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:28,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:29,065 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303146335d657e60549b2bc41c7563643f9b3 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303146335d657e60549b2bc41c7563643f9b3 2023-03-14 21:56:29,066 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:29,118 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230314/family1/000e0cf0da9643599df90153ac3542af first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-14 21:56:29,119 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f., hostname=jenkins-hbase4.apache.org,35143,1678830821197, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230314/family1/000e0cf0da9643599df90153ac3542af}] 2023-03-14 21:56:29,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(763): Validating hfile at hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230314/family1/000e0cf0da9643599df90153ac3542af for inclusion in 0446aeb6869a502bba9844ed82b4594f/family1 2023-03-14 21:56:29,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-14 21:56:29,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-14 21:56:29,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegion(2446): Flush status journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:29,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230314/family1/000e0cf0da9643599df90153ac3542af to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__i9onk97u0gtmrovmd0heefuemmb4eroas6rd0hph4ktgh5dkt990f3it3g27ul89/family1/000e0cf0da9643599df90153ac3542af 2023-03-14 21:56:29,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__i9onk97u0gtmrovmd0heefuemmb4eroas6rd0hph4ktgh5dkt990f3it3g27ul89/family1/000e0cf0da9643599df90153ac3542af as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/30e3143f463c4af9a91db9e07d0ed294_SeqId_28_ 2023-03-14 21:56:29,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(849): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__i9onk97u0gtmrovmd0heefuemmb4eroas6rd0hph4ktgh5dkt990f3it3g27ul89/family1/000e0cf0da9643599df90153ac3542af into 0446aeb6869a502bba9844ed82b4594f/family1 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/30e3143f463c4af9a91db9e07d0ed294_SeqId_28_ - updating store file list. 2023-03-14 21:56:29,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(882): Loaded HFile hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/30e3143f463c4af9a91db9e07d0ed294_SeqId_28_ into 0446aeb6869a502bba9844ed82b4594f/family1 2023-03-14 21:56:29,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.HStore(855): Successfully loaded hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__i9onk97u0gtmrovmd0heefuemmb4eroas6rd0hph4ktgh5dkt990f3it3g27ul89/family1/000e0cf0da9643599df90153ac3542af into 0446aeb6869a502bba9844ed82b4594f/family1 (new location: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/family1/30e3143f463c4af9a91db9e07d0ed294_SeqId_28_) 2023-03-14 21:56:29,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35143] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/staging/jenkins__testMajorCompactionFromAdmin__i9onk97u0gtmrovmd0heefuemmb4eroas6rd0hph4ktgh5dkt990f3it3g27ul89/family1/000e0cf0da9643599df90153ac3542af 2023-03-14 21:56:29,181 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230314 table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:29,181 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:29,182 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303141d947ab2b79e4c31a0906bf6f126c6a8 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303141d947ab2b79e4c31a0906bf6f126c6a8 2023-03-14 21:56:29,183 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145afaec6919964e8ca38fcf58e2d88b4c to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145afaec6919964e8ca38fcf58e2d88b4c 2023-03-14 21:56:29,184 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145d59b0f6965d4ad1bab65fae03d184e5 to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303145d59b0f6965d4ad1bab65fae03d184e5 2023-03-14 21:56:29,185 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314fbe82d5b41b347ea870a5299a86572cb to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314fbe82d5b41b347ea870a5299a86572cb 2023-03-14 21:56:29,185 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-14 21:56:29,185 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230314' 2023-03-14 21:56:29,186 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230314 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:29,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-14 21:56:29,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-14 21:56:29,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del] 2023-03-14 21:56:29,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:29,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230314ba556930e76745488d68b790e9b21969_del 2023-03-14 21:56:29,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-14 21:56:29,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del] 2023-03-14 21:56:29,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:29,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303147859a08f23f349c89a22f6b73f70c8dc_del 2023-03-14 21:56:29,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-14 21:56:29,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del] 2023-03-14 21:56:29,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(360): Archiving compacted files. 2023-03-14 21:56:29,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del to hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230314efb67be918db4b588c0b30d333410fda_del 2023-03-14 21:56:29,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-14 21:56:29,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678830825455] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-14 21:56:29,192 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-14 21:56:29,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.3900 sec 2023-03-14 21:56:29,213 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:29,215 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:29,217 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-14 21:56:29,218 DEBUG [Listener at localhost/45975] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-14 21:56:29,222 INFO [Listener at localhost/45975] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-14 21:56:29,258 INFO [Listener at localhost/45975] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 261), OpenFileDescriptor=628 (was 657), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=243 (was 254), ProcessCount=168 (was 169), AvailableMemoryMB=3144 (was 3167) 2023-03-14 21:56:29,259 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7defec53 to 127.0.0.1:58882 2023-03-14 21:56:29,260 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,260 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-14 21:56:29,260 INFO [Listener at localhost/45975] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-14 21:56:29,260 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0629018b to 127.0.0.1:58882 2023-03-14 21:56:29,260 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,260 DEBUG [Listener at localhost/45975] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-14 21:56:29,261 DEBUG [Listener at localhost/45975] util.JVMClusterUtil(257): Found active master hash=1029184251, stopped=false 2023-03-14 21:56:29,261 INFO [Listener at localhost/45975] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:56:29,262 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-14 21:56:29,262 INFO [Listener at localhost/45975] procedure2.ProcedureExecutor(629): Stopping 2023-03-14 21:56:29,262 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-14 21:56:29,262 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:56:29,262 DEBUG [Listener at localhost/45975] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x664ce928 to 127.0.0.1:58882 2023-03-14 21:56:29,263 DEBUG [Listener at localhost/45975] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-14 21:56:29,263 INFO [Listener at localhost/45975] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35143,1678830821197' ***** 2023-03-14 21:56:29,263 INFO [Listener at localhost/45975] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-14 21:56:29,263 INFO [RS:0;jenkins-hbase4:35143] regionserver.HeapMemoryManager(220): Stopping 2023-03-14 21:56:29,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-14 21:56:29,264 INFO [RS:0;jenkins-hbase4:35143] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-14 21:56:29,264 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-14 21:56:29,264 INFO [RS:0;jenkins-hbase4:35143] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-14 21:56:29,264 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for a6dde8759414eded6aef7ffc9c489ceb 2023-03-14 21:56:29,264 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for d3dd811eb90158aef21c530fc2c87deb 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for df07f4658e3072f4f5757b3e959f3960 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for aec273c00f5dacbd3228bda8e96041d0 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 068a37104fe03c938dbf739e3e0c8aff 2023-03-14 21:56:29,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a6dde8759414eded6aef7ffc9c489ceb, disabling compactions & flushes 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 8940ea8e9d4ee2ca9167816f0547cdcb 2023-03-14 21:56:29,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for d56bd8e4f6c9c696d51e6aa54aa60658 2023-03-14 21:56:29,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 0446aeb6869a502bba9844ed82b4594f 2023-03-14 21:56:29,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. after waiting 0 ms 2023-03-14 21:56:29,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for e05811c07f91e926119a25e5280a90f3 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 3449ac5f17f4db041a20b3d5e6f39a7f 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 1720ef3acd9355f39042767aa58726e7 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for cc098ced3bb3c092595d9641c9f57be0 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for cdbd6374c3e2fe4a973c440ad92e967b 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for a9c2e66ef8dbc6ac6ba9e46acd352171 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for c0a565a05ac628117ca5695f343304aa 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 74aa35ec7f44b881e5d675726ee27c9b 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 664facdeb00d26e025c40b36746b659d 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 23a9524a7f137f65decd5c4c40a09474 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for b326da14072a6bfe886e8eabccb45fdf 2023-03-14 21:56:29,265 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:29,265 DEBUG [RS:0;jenkins-hbase4:35143] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28c65a01 to 127.0.0.1:58882 2023-03-14 21:56:29,266 DEBUG [RS:0;jenkins-hbase4:35143] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,266 INFO [RS:0;jenkins-hbase4:35143] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-14 21:56:29,266 INFO [RS:0;jenkins-hbase4:35143] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-14 21:56:29,266 INFO [RS:0;jenkins-hbase4:35143] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-14 21:56:29,266 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-14 21:56:29,267 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-14 21:56:29,267 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-14 21:56:29,267 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-14 21:56:29,267 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-14 21:56:29,267 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-14 21:56:29,267 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-14 21:56:29,267 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-14 21:56:29,268 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1505): Waiting on 0446aeb6869a502bba9844ed82b4594f, 068a37104fe03c938dbf739e3e0c8aff, 1588230740, 1720ef3acd9355f39042767aa58726e7, 23a9524a7f137f65decd5c4c40a09474, 3449ac5f17f4db041a20b3d5e6f39a7f, 664facdeb00d26e025c40b36746b659d, 74aa35ec7f44b881e5d675726ee27c9b, 8940ea8e9d4ee2ca9167816f0547cdcb, a6dde8759414eded6aef7ffc9c489ceb, a9c2e66ef8dbc6ac6ba9e46acd352171, aec273c00f5dacbd3228bda8e96041d0, b326da14072a6bfe886e8eabccb45fdf, c0a565a05ac628117ca5695f343304aa, cc098ced3bb3c092595d9641c9f57be0, cdbd6374c3e2fe4a973c440ad92e967b, d3dd811eb90158aef21c530fc2c87deb, d56bd8e4f6c9c696d51e6aa54aa60658, df07f4658e3072f4f5757b3e959f3960, e05811c07f91e926119a25e5280a90f3, f16657fdf561e3e0eb764d4e08e94891 2023-03-14 21:56:29,271 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-14 21:56:29,288 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/info/1885ff59bfc040c490ee4a50a1b8bb19 2023-03-14 21:56:29,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a6dde8759414eded6aef7ffc9c489ceb/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:56:29,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a6dde8759414eded6aef7ffc9c489ceb: 2023-03-14 21:56:29,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678830942068.a6dde8759414eded6aef7ffc9c489ceb. 2023-03-14 21:56:29,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d3dd811eb90158aef21c530fc2c87deb, disabling compactions & flushes 2023-03-14 21:56:29,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:56:29,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:56:29,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. after waiting 0 ms 2023-03-14 21:56:29,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:56:29,292 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1885ff59bfc040c490ee4a50a1b8bb19 2023-03-14 21:56:29,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d3dd811eb90158aef21c530fc2c87deb/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-14 21:56:29,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:56:29,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d3dd811eb90158aef21c530fc2c87deb: 2023-03-14 21:56:29,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678830878374.d3dd811eb90158aef21c530fc2c87deb. 2023-03-14 21:56:29,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing df07f4658e3072f4f5757b3e959f3960, disabling compactions & flushes 2023-03-14 21:56:29,301 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:56:29,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:56:29,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. after waiting 0 ms 2023-03-14 21:56:29,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:56:29,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/df07f4658e3072f4f5757b3e959f3960/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-14 21:56:29,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for df07f4658e3072f4f5757b3e959f3960: 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678830916313.df07f4658e3072f4f5757b3e959f3960. 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aec273c00f5dacbd3228bda8e96041d0, disabling compactions & flushes 2023-03-14 21:56:29,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. after waiting 0 ms 2023-03-14 21:56:29,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:56:29,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/aec273c00f5dacbd3228bda8e96041d0/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-14 21:56:29,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aec273c00f5dacbd3228bda8e96041d0: 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678830862880.aec273c00f5dacbd3228bda8e96041d0. 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 068a37104fe03c938dbf739e3e0c8aff, disabling compactions & flushes 2023-03-14 21:56:29,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. after waiting 0 ms 2023-03-14 21:56:29,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:56:29,327 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-14 21:56:29,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/068a37104fe03c938dbf739e3e0c8aff/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-14 21:56:29,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:56:29,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 068a37104fe03c938dbf739e3e0c8aff: 2023-03-14 21:56:29,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678830916313.068a37104fe03c938dbf739e3e0c8aff. 2023-03-14 21:56:29,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8940ea8e9d4ee2ca9167816f0547cdcb, disabling compactions & flushes 2023-03-14 21:56:29,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:56:29,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:56:29,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. after waiting 0 ms 2023-03-14 21:56:29,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:56:29,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/8940ea8e9d4ee2ca9167816f0547cdcb/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-14 21:56:29,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:56:29,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8940ea8e9d4ee2ca9167816f0547cdcb: 2023-03-14 21:56:29,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678830862880.8940ea8e9d4ee2ca9167816f0547cdcb. 2023-03-14 21:56:29,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d56bd8e4f6c9c696d51e6aa54aa60658, disabling compactions & flushes 2023-03-14 21:56:29,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:56:29,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:56:29,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. after waiting 0 ms 2023-03-14 21:56:29,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:56:29,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithWeeklyPolicy/d56bd8e4f6c9c696d51e6aa54aa60658/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-14 21:56:29,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d56bd8e4f6c9c696d51e6aa54aa60658: 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678830851486.d56bd8e4f6c9c696d51e6aa54aa60658. 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0446aeb6869a502bba9844ed82b4594f, disabling compactions & flushes 2023-03-14 21:56:29,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. after waiting 0 ms 2023-03-14 21:56:29,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:29,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/0446aeb6869a502bba9844ed82b4594f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0446aeb6869a502bba9844ed82b4594f: 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678830970607.0446aeb6869a502bba9844ed82b4594f. 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e05811c07f91e926119a25e5280a90f3, disabling compactions & flushes 2023-03-14 21:56:29,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. after waiting 0 ms 2023-03-14 21:56:29,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:56:29,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyPolicy/e05811c07f91e926119a25e5280a90f3/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-14 21:56:29,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:56:29,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e05811c07f91e926119a25e5280a90f3: 2023-03-14 21:56:29,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678830866484.e05811c07f91e926119a25e5280a90f3. 2023-03-14 21:56:29,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3449ac5f17f4db041a20b3d5e6f39a7f, disabling compactions & flushes 2023-03-14 21:56:29,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:29,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:29,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. after waiting 0 ms 2023-03-14 21:56:29,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:29,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/3449ac5f17f4db041a20b3d5e6f39a7f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3449ac5f17f4db041a20b3d5e6f39a7f: 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678830970607.3449ac5f17f4db041a20b3d5e6f39a7f. 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1720ef3acd9355f39042767aa58726e7, disabling compactions & flushes 2023-03-14 21:56:29,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. after waiting 0 ms 2023-03-14 21:56:29,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:56:29,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithMonthlyPolicy/1720ef3acd9355f39042767aa58726e7/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-14 21:56:29,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:56:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1720ef3acd9355f39042767aa58726e7: 2023-03-14 21:56:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678830838262.1720ef3acd9355f39042767aa58726e7. 2023-03-14 21:56:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cc098ced3bb3c092595d9641c9f57be0, disabling compactions & flushes 2023-03-14 21:56:29,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. after waiting 0 ms 2023-03-14 21:56:29,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:29,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionFromAdmin/cc098ced3bb3c092595d9641c9f57be0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:29,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cc098ced3bb3c092595d9641c9f57be0: 2023-03-14 21:56:29,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678830970607.cc098ced3bb3c092595d9641c9f57be0. 2023-03-14 21:56:29,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cdbd6374c3e2fe4a973c440ad92e967b, disabling compactions & flushes 2023-03-14 21:56:29,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:56:29,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:56:29,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. after waiting 0 ms 2023-03-14 21:56:29,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:56:29,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing cdbd6374c3e2fe4a973c440ad92e967b 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-14 21:56:29,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/.tmp/info/b1056425391f4975bb7e96dc66360429 2023-03-14 21:56:29,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/.tmp/info/b1056425391f4975bb7e96dc66360429 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/info/b1056425391f4975bb7e96dc66360429 2023-03-14 21:56:29,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/info/b1056425391f4975bb7e96dc66360429, entries=3, sequenceid=7, filesize=4.8 K 2023-03-14 21:56:29,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for cdbd6374c3e2fe4a973c440ad92e967b in 30ms, sequenceid=7, compaction requested=false 2023-03-14 21:56:29,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/namespace/cdbd6374c3e2fe4a973c440ad92e967b/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-14 21:56:29,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:56:29,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cdbd6374c3e2fe4a973c440ad92e967b: 2023-03-14 21:56:29,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678830824339.cdbd6374c3e2fe4a973c440ad92e967b. 2023-03-14 21:56:29,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a9c2e66ef8dbc6ac6ba9e46acd352171, disabling compactions & flushes 2023-03-14 21:56:29,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:56:29,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:56:29,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. after waiting 0 ms 2023-03-14 21:56:29,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/a9c2e66ef8dbc6ac6ba9e46acd352171/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a9c2e66ef8dbc6ac6ba9e46acd352171: 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678830942068.a9c2e66ef8dbc6ac6ba9e46acd352171. 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c0a565a05ac628117ca5695f343304aa, disabling compactions & flushes 2023-03-14 21:56:29,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. after waiting 0 ms 2023-03-14 21:56:29,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:56:29,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/ns/testMinorCompaction/c0a565a05ac628117ca5695f343304aa/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-14 21:56:29,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c0a565a05ac628117ca5695f343304aa: 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678830942068.c0a565a05ac628117ca5695f343304aa. 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74aa35ec7f44b881e5d675726ee27c9b, disabling compactions & flushes 2023-03-14 21:56:29,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. after waiting 0 ms 2023-03-14 21:56:29,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:56:29,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/74aa35ec7f44b881e5d675726ee27c9b/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-14 21:56:29,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:56:29,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74aa35ec7f44b881e5d675726ee27c9b: 2023-03-14 21:56:29,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678830893878.74aa35ec7f44b881e5d675726ee27c9b. 2023-03-14 21:56:29,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 664facdeb00d26e025c40b36746b659d, disabling compactions & flushes 2023-03-14 21:56:29,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:56:29,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:56:29,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. after waiting 0 ms 2023-03-14 21:56:29,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/664facdeb00d26e025c40b36746b659d/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-14 21:56:29,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 664facdeb00d26e025c40b36746b659d: 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678830825633.664facdeb00d26e025c40b36746b659d. 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f16657fdf561e3e0eb764d4e08e94891, disabling compactions & flushes 2023-03-14 21:56:29,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. after waiting 0 ms 2023-03-14 21:56:29,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:56:29,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testCompactionWithHFileLink/f16657fdf561e3e0eb764d4e08e94891/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-14 21:56:29,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f16657fdf561e3e0eb764d4e08e94891: 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678830916313.f16657fdf561e3e0eb764d4e08e94891. 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 23a9524a7f137f65decd5c4c40a09474, disabling compactions & flushes 2023-03-14 21:56:29,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. after waiting 0 ms 2023-03-14 21:56:29,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:29,456 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testMinorCompactionWithMonthlyPolicy/23a9524a7f137f65decd5c4c40a09474/recovered.edits/51.seqid, newMaxSeqId=51, maxSeqId=1 2023-03-14 21:56:29,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 23a9524a7f137f65decd5c4c40a09474: 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678830960013.23a9524a7f137f65decd5c4c40a09474. 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b326da14072a6bfe886e8eabccb45fdf, disabling compactions & flushes 2023-03-14 21:56:29,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. after waiting 0 ms 2023-03-14 21:56:29,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:56:29,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/default/testScannerOnBulkLoadRefHFiles/b326da14072a6bfe886e8eabccb45fdf/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-14 21:56:29,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:56:29,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b326da14072a6bfe886e8eabccb45fdf: 2023-03-14 21:56:29,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678830862880.b326da14072a6bfe886e8eabccb45fdf. 2023-03-14 21:56:29,468 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-14 21:56:29,668 DEBUG [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-14 21:56:29,704 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/rep_barrier/f10b64d8e5d94a4ea9b192d46bf82c52 2023-03-14 21:56:29,709 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f10b64d8e5d94a4ea9b192d46bf82c52 2023-03-14 21:56:29,717 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/table/c34877f253b94cafb7ede9d7eb99e28a 2023-03-14 21:56:29,722 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c34877f253b94cafb7ede9d7eb99e28a 2023-03-14 21:56:29,723 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/info/1885ff59bfc040c490ee4a50a1b8bb19 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/info/1885ff59bfc040c490ee4a50a1b8bb19 2023-03-14 21:56:29,727 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1885ff59bfc040c490ee4a50a1b8bb19 2023-03-14 21:56:29,727 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/info/1885ff59bfc040c490ee4a50a1b8bb19, entries=226, sequenceid=130, filesize=37.9 K 2023-03-14 21:56:29,728 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/rep_barrier/f10b64d8e5d94a4ea9b192d46bf82c52 as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/rep_barrier/f10b64d8e5d94a4ea9b192d46bf82c52 2023-03-14 21:56:29,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f10b64d8e5d94a4ea9b192d46bf82c52 2023-03-14 21:56:29,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/rep_barrier/f10b64d8e5d94a4ea9b192d46bf82c52, entries=1, sequenceid=130, filesize=5.0 K 2023-03-14 21:56:29,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/.tmp/table/c34877f253b94cafb7ede9d7eb99e28a as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/table/c34877f253b94cafb7ede9d7eb99e28a 2023-03-14 21:56:29,748 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c34877f253b94cafb7ede9d7eb99e28a 2023-03-14 21:56:29,748 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/table/c34877f253b94cafb7ede9d7eb99e28a, entries=27, sequenceid=130, filesize=6.6 K 2023-03-14 21:56:29,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 483ms, sequenceid=130, compaction requested=false 2023-03-14 21:56:29,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-14 21:56:29,756 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-14 21:56:29,756 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-14 21:56:29,756 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-14 21:56:29,756 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-14 21:56:29,868 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35143,1678830821197; all regions closed. 2023-03-14 21:56:29,875 DEBUG [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/oldWALs 2023-03-14 21:56:29,875 INFO [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35143%2C1678830821197.meta:.meta(num 1678830824061) 2023-03-14 21:56:29,879 DEBUG [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/oldWALs 2023-03-14 21:56:29,879 INFO [RS:0;jenkins-hbase4:35143] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35143%2C1678830821197:(num 1678830823086) 2023-03-14 21:56:29,879 DEBUG [RS:0;jenkins-hbase4:35143] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,879 INFO [RS:0;jenkins-hbase4:35143] regionserver.LeaseManager(133): Closed leases 2023-03-14 21:56:29,879 INFO [RS:0;jenkins-hbase4:35143] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-14 21:56:29,880 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-14 21:56:29,880 INFO [RS:0;jenkins-hbase4:35143] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35143 2023-03-14 21:56:29,890 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35143,1678830821197 2023-03-14 21:56:29,890 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-14 21:56:29,890 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@67953702 rejected from java.util.concurrent.ThreadPoolExecutor@63aabf28[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:29,891 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-14 21:56:29,891 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35143,1678830821197] 2023-03-14 21:56:29,891 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5927dda0 rejected from java.util.concurrent.ThreadPoolExecutor@63aabf28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:29,891 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35143,1678830821197; numProcessing=1 2023-03-14 21:56:29,893 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35143,1678830821197 already deleted, retry=false 2023-03-14 21:56:29,893 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35143,1678830821197 expired; onlineServers=0 2023-03-14 21:56:29,893 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40679,1678830820017' ***** 2023-03-14 21:56:29,893 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-14 21:56:29,893 DEBUG [M:0;jenkins-hbase4:40679] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c6510c0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-14 21:56:29,893 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40679,1678830820017 2023-03-14 21:56:29,893 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40679,1678830820017; all regions closed. 2023-03-14 21:56:29,893 DEBUG [M:0;jenkins-hbase4:40679] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-14 21:56:29,894 DEBUG [M:0;jenkins-hbase4:40679] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-14 21:56:29,894 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-14 21:56:29,894 DEBUG [M:0;jenkins-hbase4:40679] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-14 21:56:29,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678830822778] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678830822778,5,FailOnTimeoutGroup] 2023-03-14 21:56:29,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678830822779] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678830822779,5,FailOnTimeoutGroup] 2023-03-14 21:56:29,894 INFO [M:0;jenkins-hbase4:40679] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-14 21:56:29,894 INFO [M:0;jenkins-hbase4:40679] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-14 21:56:29,894 INFO [M:0;jenkins-hbase4:40679] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-14 21:56:29,895 DEBUG [M:0;jenkins-hbase4:40679] master.HMaster(1512): Stopping service threads 2023-03-14 21:56:29,895 INFO [M:0;jenkins-hbase4:40679] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-14 21:56:29,895 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-14 21:56:29,895 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-14 21:56:29,895 ERROR [M:0;jenkins-hbase4:40679] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-14 21:56:29,895 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-14 21:56:29,896 INFO [M:0;jenkins-hbase4:40679] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-14 21:56:29,896 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-14 21:56:29,896 DEBUG [M:0;jenkins-hbase4:40679] zookeeper.ZKUtil(398): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-14 21:56:29,896 WARN [M:0;jenkins-hbase4:40679] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-14 21:56:29,896 INFO [M:0;jenkins-hbase4:40679] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-14 21:56:29,896 INFO [M:0;jenkins-hbase4:40679] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-14 21:56:29,897 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-14 21:56:29,897 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:56:29,897 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:56:29,897 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-14 21:56:29,897 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:56:29,897 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.95 KB heapSize=599.12 KB 2023-03-14 21:56:29,992 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-14 21:56:29,992 INFO [RS:0;jenkins-hbase4:35143] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35143,1678830821197; zookeeper connection closed. 2023-03-14 21:56:29,992 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@21a1628e rejected from java.util.concurrent.ThreadPoolExecutor@63aabf28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:29,992 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): regionserver:35143-0x100e50541210001, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-14 21:56:29,992 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@212acddb rejected from java.util.concurrent.ThreadPoolExecutor@63aabf28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:29,992 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5d524901] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5d524901 2023-03-14 21:56:29,993 INFO [Listener at localhost/45975] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-14 21:56:30,170 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-14 21:56:30,306 INFO [M:0;jenkins-hbase4:40679] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.95 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e9345cdf174c494f939820b4d8ab6beb 2023-03-14 21:56:30,309 INFO [M:0;jenkins-hbase4:40679] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9345cdf174c494f939820b4d8ab6beb 2023-03-14 21:56:30,310 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e9345cdf174c494f939820b4d8ab6beb as hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e9345cdf174c494f939820b4d8ab6beb 2023-03-14 21:56:30,313 INFO [M:0;jenkins-hbase4:40679] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9345cdf174c494f939820b4d8ab6beb 2023-03-14 21:56:30,313 INFO [M:0;jenkins-hbase4:40679] regionserver.HStore(1080): Added hdfs://localhost:39547/user/jenkins/test-data/cca07b6c-1cdf-c92d-7f19-669f4a5e2a63/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e9345cdf174c494f939820b4d8ab6beb, entries=123, sequenceid=1011, filesize=33.3 K 2023-03-14 21:56:30,314 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegion(2948): Finished flush of dataSize ~512.95 KB/525261, heapSize ~599.10 KB/613480, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 417ms, sequenceid=1011, compaction requested=false 2023-03-14 21:56:30,315 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-14 21:56:30,315 DEBUG [M:0;jenkins-hbase4:40679] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-14 21:56:30,317 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-14 21:56:30,317 INFO [M:0;jenkins-hbase4:40679] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-14 21:56:30,318 INFO [M:0;jenkins-hbase4:40679] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40679 2023-03-14 21:56:30,321 DEBUG [M:0;jenkins-hbase4:40679] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,40679,1678830820017 already deleted, retry=false 2023-03-14 21:56:30,423 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-14 21:56:30,423 INFO [M:0;jenkins-hbase4:40679] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40679,1678830820017; zookeeper connection closed. 2023-03-14 21:56:30,423 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@31a8d85d rejected from java.util.concurrent.ThreadPoolExecutor@72d2f450[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:30,423 DEBUG [Listener at localhost/45975-EventThread] zookeeper.ZKWatcher(600): master:40679-0x100e50541210000, quorum=127.0.0.1:58882, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-14 21:56:30,423 ERROR [Listener at localhost/45975-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7b5e7926 rejected from java.util.concurrent.ThreadPoolExecutor@72d2f450[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-14 21:56:30,425 WARN [Listener at localhost/45975] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-14 21:56:30,427 INFO [Listener at localhost/45975] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-14 21:56:30,533 WARN [BP-1644408315-172.31.14.131-1678830816501 heartbeating to localhost/127.0.0.1:39547] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-14 21:56:30,533 WARN [BP-1644408315-172.31.14.131-1678830816501 heartbeating to localhost/127.0.0.1:39547] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1644408315-172.31.14.131-1678830816501 (Datanode Uuid ea84138e-3091-4fa6-8b1e-81f8e4e08e46) service to localhost/127.0.0.1:39547 2023-03-14 21:56:30,535 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/dfs/data/data1/current/BP-1644408315-172.31.14.131-1678830816501] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-14 21:56:30,535 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9e232f6f-99e6-d682-7eec-cdefcbd7e80b/cluster_afccc979-e832-af34-d2b7-8700af2e25f5/dfs/data/data2/current/BP-1644408315-172.31.14.131-1678830816501] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-14 21:56:30,560 INFO [Listener at localhost/45975] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-14 21:56:30,697 INFO [Listener at localhost/45975] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-14 21:56:30,807 INFO [Listener at localhost/45975] hbase.HBaseTestingUtility(1293): Minicluster is down